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1.  Document  Overview 

This  Final  Report  describes  BBN’s  IMMERSE  project,  funded  as  part  of  the  Defense  Advanced 
Research  Projects  Agency  (DARPA)  Strategic  Social  Interaction  Modules  (SSIM)  program. 

Section  1  Document  Overview  provides  an  overview  of  the  document. 

Section  2  Project  Overview  provides  an  overview  of  the  IMMERSE  project,  its  purpose,  the 
concept  of  operations  for  the  resulting  capability,  and  a  description  of  the  team. 

Section  3  Accomplishments  &  Major  Milestones  describes  the  major  demonstrations,  system 
deployments,  and  publications  and  presentations  made  as  part  of  the  IMMERSE  effort. 

Section  4  Simulator  System  Description  describes  the  functional  details  of  the  IMMERSE 
Social  Simulator,  including  the  system  architecture  and  major  functional  components. 

Section  5  Expressive  Artificial  Intelligence  for  Social  Immersion  describes  the  technologies 
developed  for  supporting  the  IMMERSE  Social  Simulator  with  its  unscripted  social  interaction 
with  virtual  characters. 

Section  6  Social  Pedagogy  describes  how  pedagogy  is  manifest  within  the  IMMERSE  Social 
Simulator,  including  the  design  of  social  encounters,  social  simulation  as  a  training  vehicle,  the 
social  coach,  the  role  of  the  pedagogic  director,  and  after  action  review. 

Section  7  Additional  Knowledge  Products  briefly  describes  two  additional  knowledge  products 
developed  under  the  IMMERSE  project:  a  Gap  Analysis  Study  by  MacroCognition,  seeking  to 
identify  the  gaps  between  the  Good  Stranger  (GS)  skills  needed  and  those  possessed  by  the 
military  personnel,  and  research  by  SIFT  to  determine  how  to  best  represent  “body  behaviors” 
within  the  framework  of  the  Etiquette  Engine,  previously  developed  as  an  adjunct  to  DARPA’s 
DARWARS  projects. 

Section  8  Lessons  Learned  details  a  series  of  lessons  learned  in  developing  the  IMMERSE 
Social  Simulator. 

Section  9  Unanswered  Questions  about  Social  Interaction  with  Virtual  Characters 

highlights  a  number  of  outstanding  issues  involving  a  player  interacting  with  virtual  characters. 

Section  10  Recommendations  on  the  Way  Ahead  discusses  areas  for  further  research  in  order 
to  expand  the  state-of-the-art  in  social  simulation. 

Section  11  Conclusion  looks  ahead  to  the  Army’s  “Seventh  Warfighting  Requirement”. 

Section  12  Acknowledgements  recognizes  the  many  contributors  that  have  added  to  the  success 
of  the  IMMERSE  project. 

Section  13  Appendices  contains  a  list  of  symbols,  abbreviations,  and  acronyms,  document 
references,  a  subsection  on  system  development  history,  a  subsection  on  the  software  developed 
to  facilitate  visualization  of  the  virtual  social  experience,  and  reports  from  SIFT  and 
MacroCognition . 


Use  or  disclosure  of  the  data  contained  on  this  page  is  subject  to  the  restrictions  on  the  title  page  of  this  report. 


Page  1 


IMMERSE  Final  Report 

Contract  Number:  W91  INF-1 2-C-0002 


Raytheon  BBN  Technologies 


2.  Project  Overview 

2.1  Purpose 

The  DARPA  SSIM  program  was  designed  to  address  the  challenge  of  training  our  warfighters  to 
successfully  engage  with  people  from  other  cultures  in  order  to  accomplish  missions  where 
success  hinges  on  social  competence.  As  part  of  this  program,  BBN’s  IMMERSE  effort  aimed  to 
demonstrate  the  possibility  of  conducting  training  using  an  immersive,  simulation-based  training 
system,  called  the  Social  Simulator,  to  create  a  mentored,  social-interaction  practice 
environment.  Technical  challenges  included  developing  interactive  synthetic  social  situations 
populated  by  robust,  believable  agents  and  providing  a  synthetic  coach  to  deliver  feedback  and 
guidance.  In  the  Social  Simulator,  trainees  can  expect  to  interact  with  simulated  characters  the 
same  way  they  interact  with  other  people,  through  speech,  gesture,  comportment,  and  other 
forms  of  nonverbal  communication. 

The  original  vision  for  DARPA’ s  SSIM  program  was  to  understand,  measure  and  teach  the 
ability  to  be  a  “good  stranger”  -  to  interact  effectively  with  people  from  other  cultures  without 
trying  to  be  accepted  as  a  member  of  that  culture.  Because  the  expected  SSIM  learners  were  to 
come  from  the  military  and  law  enforcement,  SSIM  focused  on  interactions  with  the  potential  for 
kinetic  escalation  (violence  breaking  out)  and  where  the  learner  was  often  trying  to  “gain 
compliance”  —  get  members  of  another  culture  to  do  something  or  provide  information.  In 
keeping  with  the  good  stranger  vision,  the  Social  Simulator  does  not  teach  culture-specific  skills, 
knowledge  and  behavior;  rather,  it  aims  to  sensitize  the  learner  to  anticipate  and  perceive  culture- 
specific  behavior  and  adapt  to  it  in  the  field.  The  Social  Simulator  is  composed  of  two  elements: 
MiBA  and  IMMERSE.  MiBA  senses  and  recognizes  learner  actions  and  passes  them  to 
IMMERSE,  which  interprets  the  actions  in  a  simulated  social  environment. 

Conceptually,  the  design  for  the  Social  Simulator  began  with  the  following  notion:  if  social 
competence  is  a  skill,  then  it  can  be  trained  through  relevant  practice  with  feedback.  Practice  of 
real-time,  embodied,  cognitively  rich  skills  is  made  more  comprehensive  by  embodied 
interaction,  real-time  dynamism  and  socio-cognitive  richness.  Thus  the  virtual  characters  (VCs) 
that  inhabit  IMMERSE  must  be  able  to  detect,  interpret  and  react  to  the  full  body  behaviors  of 
the  learner  in  real  time.  They  must  then  behave  as  if  they  are  part  of  the  many  overlapping  social 
interactions  that  make  up  multi-person,  social  settings. 

Besides  this  basic  assumption,  the  design  was  guided  by  two  secondary  principles: 

1.  Performance  Hypothesis:  High  fidelity  simulation  is  not  enough.  The  simulated 
characters  must  perform  social  actions  (as  in  the  kind  of  thing  actors  and  actresses  do  to 
make  their  behavior  meaningful  to  an  audience)  in  order  to  enable  pedagogically 
effective  social  interaction. 

2.  Social  Coaching  Hypothesis:  Coaching  social  competence  is  especially  sensitive  to  the 
social  competence  of  the  coach. 

The  IMMERSE  portion  of  the  SSIM  effort  is  an  engineering  effort  aimed  at  building  a  prototype 
rather  than  a  validation-oriented  experimental  effort.  It  has  attempted  to  demonstrate  that  a 
system  can  be  designed  and  built  to  effectively  exploit  these  hypotheses  and  to  enable 
experiments  to  test  them.  That  said,  there  is  real  benefit  to  be  gained  if  systems  based  on  these 
conjectures  and  the  Social  Simulator  prototype  can  be  built  and  fielded  -  the  simplest  being  the 
existence  of  a  scalable  technology  for  delivering  social  training,  given  that  the  current  best 
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practice — human  role-play  based  training — is  expensive,  logistically  cumbersome,  and  prone  to 
systematic  weaknesses  such  as  role  player  fatigue. 


2.2  Concept  of  Operations 


Interacting  with  the  Social  Simulator  is  meant  to  be  a  natural  experience  for  the  learner.  The 
learner  stands  in  front  of  a  large  display,  such  as  a  70”  screen,  equipped  with  a  microphone  and  a 
Kinect  2  sensor.  Figure  1  illustrates  the  physical  layout  of  the  environment.  The  learner  is  free  to 
move  around  in  front  of  the  display  within  an  11.5  ft.  deep  by  11.5  ft.  wide  region,  as  constrained 
by  Kinect  2  sensor.  We  utilize  a  noise  cancelling,  wireless  microphone  to  overcome  background 
noise,  but  the  Kinect  2  sensor  also  offers  a  built-in  microphone  array,  which  can  be  utilized  to 
reduce  hardware  requirements.  Depending  on  the  scenario,  the  learner  may  employ  a  simulated 
weapon  that  contains  an  embedded  game  controller.  Figure  2  illustrates  a  user  interacting  with 
the  system. 
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Figure  1:  Top-down  view  of  Social  Simulator  physical  layout. 


Figure  2:  Trainee  interacting  with  the  Social  Simulator. 


A  typical  session  with  a  learner  involves  short  system  configuration  phase  and  a  short  mission 
briefing,  after  which  the  learner  moves  into  the  scenario,  which  lasts  on  the  order  of  two  to  ten 
minutes.  During  the  scenario,  the  learner  interacts  with  social  virtual  characters,  including  a 
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virtual  coach,  which  provides  guidance  and  feedback  as  appropriate.  After  the  scenario,  the 
system  provides  an  after  action  review  (AAR)  describing  effective  and  ineffective  learner 
behaviors  and  suggesting  what  the  learner  might  do  to  perform  better  in  the  scenario. 

2.3  Team 

The  IMMERSE  project  team  (See  Figure  3)  included  Raytheon  BBN  Technologies,  whose 
responsibilities  include  integration,  the  virtual  environment,  and  coaching;  the  Center  for  Games 
and  Playable  Media  at  the  University  of  California,  Santa  Cruz,  who  focused  on  social 
simulation,  behavior  modeling,  and  performance;  Fire  Hose  Games,  who  developed  the  art  and 
non-dialog  audio  assets  necessary  for  the  virtual  environment;  MacroCognition  LLC  who 
assisted  in  the  design  of  social  interaction  skills  training,  and  acted  as  a  liaison  with  the  social 
scientists  on  co-contractor  teams;  Smart  Information  Flow  Technologies  (SIFT),  who  designed 
capabilities  for  assessing  etiquette  perceptions  and  misperceptions  among  culturally  diverse 
participants;  and  Dr.  Alan  Collins,  who  served  as  a  senior  advisor  on  cognitive  apprenticeship, 
situated  cognition,  and  tutoring. 
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Figure  3:  IMMERSE  Project  Team 
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3.  Accomplishments  and  Major  Milestones 

Figure  4  depicts  the  major  milestones  and  events  that  defined  the  IMMERSE  project.  Semi¬ 
annual  PI  meetings  provided  the  most  obvious  yardstick  for  measuring  progress.  With  the 
exception  of  the  kickoff  meeting  in  October  2011,  each  PI  meeting  became  an  occasion  for 
demonstrating  progress  by  showcasing  a  new  interactive  scenario  -  highlighting  different  types 
of  social  encounters  -  or  technology.  Section  3.1  describes  the  experiences  offered  by  each  of  the 
five  demonstation  scenarios.  In  the  final  year  of  the  project,  working  systems  were  deployed  to 
three  sites,  as  described  below  in  Section  3.2.  Section  3.3  lists  publications,  presentations  and 
panel  participation. 
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Figure  4:  IMMERSE  Timeline 
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3.1  Demonstrations 

3.1.1  Checkpoint  Demonstration 


Figure  5:  Screenshot  from  the  Checkpoint  Scenario 


Phase  1  of  the  IMMERSE  effort  was  largely  about  assembling  existing  components  and  creating 
something  new  with  the  composition:  a  functioning  Social  Simulation  Architecture.  In  the  first 
year,  we  bootstrapped  the  process  by  developing  a  scenario  focused  on  a  type  of  checkpoint 
commonly  known  as  an  entry  control  point,  at  a  food  distribution  warehouse.  This  scenario  was 
designed  to  illustrate  that: 

1.  Trainees  are  immersed  in  an  ongoing  social  milieu. 

2.  VCs  interact  socially  with  each  other,  and  with  the  trainee. 

3.  The  trainee  is  standing,  unencumbered,  and  interacts  with  the  VCs  using  normal  gestures, 
poses,  facial  expressions,  and  speech  (including  paralinguistic  features).  Actions  are 
interpreted  based  on  the  social  context  in  which  they  occur  and  the  understanding  of 
individual  VCs. 

4.  Social  interactions  are  not  scripted;  they  play  out  according  to  specific  rules  of  social 
behavior  applied  to  the  evolving,  shared  social  context  and  the  particular  individuals 
(VCs)  involved  in  the  exchange. 

5.  Against  this  realistic  social  backdrop,  a  simulated  mentor  shapes  the  learning  experience 
and  provides  guidance  and  feedback  to  the  trainee. 

6.  The  training  focuses  on  improving  social  skills,  especially  those  of  value  in  cross-cultural 
settings.  These  include  meta-cultural  skills  like  observation,  imitation,  experimentation, 
de-centering  (seeing  events  through  another’s  eyes);  self-monitoring  skills  like  self- 
awareness,  emotional  control  and  other  forms  of  self-regulation;  and  encounter 
management  skills  like  understanding  the  structure  of  encounters,  recognizing  and 
repairing  communication  failures,  and  techniques  for  escalation  and  de-escalation. 
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7.  These  skills  need  to  be  adapted  and  applied  in  a  variety  of  environmental  and  social 
conditions;  e.g.,  social  setting,  stress  level,  number  of  participants,  time  pressure, 
language  ability,  cultural  background,  etc. 

In  the  Checkpoint  scenario,  the  player  is  in  charge  of  checking  the  identification  of  people  who 
want  to  enter  a  Non-govemmental  organization  (NGO)  facility,  located  in  the  outskirts  of  a 
foreign  village. 

•  At  first  a  British  soldier  walks  up,  and  hands  the  player  an  identification  card.  The  player 
needs  to  hold  the  ID  up  to  their  face,  and  then  hand  it  back  to  the  soldier,  who  then  thanks  the 
player  and  walks  past  the  player. 

•  A  few  local  civilians  are  next  in  line,  but  the  man  at  the  front  of  the  line  does  not  approach. 
The  player  has  to  beckon  for  the  man  to  approach,  who  gestures  and  speaks  in  their  local 
language. 

•  The  player  may  (repeatedly)  gesture  and  speak  to  try  to  communicate  that  an  ID  is  required. 
The  man  seems  to  not  understand,  and  gets  frustrated.  He  eventually  turns  to  his  friends 
behind  him,  who  seem  upset  as  well. 

•  The  player  can  gesture  for  the  man  to  step  aside,  and  walk  over  to  a  different  place  to  stand. 
The  man  reluctantly  complies. 

3.1.2  Lost  Interpreter  Demonstration 


Figure  6:  Screenshot  from  the  Lost  Interpreter  Scenario 


Considerably  more  ambitious  than  the  Checkpoint  scenario,  Lost  Interpreter  focused  on  group 
dynamics,  in  particular  the  social  conventions  that  determine  how  a  small  number  of  people  in 
conversation  recognize  and  accommodate  to  a  new  member  joining  the  group.  We  also  explored 
techniques  for  player  locomotion  within  the  scene,  by  introducing  a  small  handheld  controller, 
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and  the  intricacies  of  transferring  objects  -  a  photograph  -  between  the  real  and  virtual  world. 
Finally,  the  scenario  presented  an  explicit  goal  -  find  the  interpreter  -  but  allowed  the  player  free 
reign  for  how  to  accomplish  this. 

The  Player  is  in  a  foreign  village  marketplace,  holding  a  photograph  of  an  interpreter  they  have 
been  told  to  meet.  There  are  various  groups  of  people  standing  throughout  the  marketplace.  The 
player  has  free  navigation,  via  a  handheld  wireless  controller/joystick. 

The  following  interactions  are  possible  in  Lost  Interpreter: 

•  The  player  approaches  the  closest  group,  who  smile  and  seem  friendly,  and  open  up  their 
circle  to  include  the  player. 

•  The  player  can  show  and  point  at  the  photo.  One  VC  will  ask  for  it,  and  the  player  can  hand  it 
to  them.  They  will  pass  the  photo  around,  looking  at  it.  One  VC  will  point  toward  another 
group  of  three  men  nearby. 

•  The  player  can  gesture  for  the  photo  back,  and  get  it,  and  then  walk  over  to  the  group  of  three 
men. 

•  These  men  look  reticent  when  the  player  approaches.  When  asked  about  the  photo,  one 
seems  to  recognize  it  and  looks  anxious,  but  the  senior  man  in  the  group  mutters  something 
that  quiets  the  other  two.  Further  asking  about  the  photo  seems  to  annoy  the  senior  man. 

•  The  player  has  to  make  a  stronger  emotional  appeal  gesture  to  the  senior  man,  to  get  him  to 
point  toward  the  fountain  in  the  center  of  the  square,  where  the  player  can  see  the  interpreter 
off  in  the  distance. 

•  As  the  player  moves  toward  the  fountain,  the  interpreter  gets  lost  in  the  crowd. 

3.1.3  Breaking  Bread  Demonstration 


Figure  7:  Screenshot  from  the  Breaking  Bread  Scenario 


Breaking  Bread  was  modeled  after  an  actual  event.  It  was  designed  to  situate  the  Player  in  a 
familiar  mission  (knock  and  search)  with  a  high  probability  of  escalation  in  order  to  emphasize 
the  ever-present  concern  for  balancing  tact  and  tactics  in  social  situations.  This  scenario 
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introduced  the  notion  of  Social  Games  (see  Section  5.1  below)  as  the  mechanism  for  guiding 
VC-VC  and  VC-Player  interactions,  which  allowed  for  greater  emotional  breadth  in  the  VC 
behaviors.  We  also  gave  the  player  a  weapon  -  a  plastic  AR-15  -  to  add  additional  affordances 
for  the  player  to  consider. 

The  Player  is  introduced  to  the  scenario  as  follows: 

•  You  are  the  leader  of  a  US  team  that  has  just  started  working  alongside  and  training  a  local 
national  (LN)  military  unit.  The  primary  US  role  in  this  region  is  wide  area  security.  You 
deployed  on  short  notice  and  arrived  in  the  region  with  very  little  background  knowledge  of 
the  culture  or  the  language.  But  so  far  things  are  going  well. 

•  You’re  currently  on  a  patrol  with  the  LN  unit  to  sweep  (knock  and  search)  several  houses  in 
a  village  that  are  suspected  hiding  places  for  contraband  -  emergency  medical  supplies  that 
did  not  arrive  at  their  intended  destination.  People  tend  to  keep  weapons  in  their  homes,  but 
the  environment  is  generally  viewed  as  permissive. 

•  The  LN  team  has  the  lead;  you  are  here  to  observe,  advise,  and  assist.  This  is  only  your 
second  patrol  with  your  local  national  counterpart,  and  the  first  time  you  have  been  in  this 
village.  You  understand  that  the  LN  soldier  (LNS)  is  from  a  neighboring  village  and  he 
speaks  almost  no  English.  His  name  is  Basuki. 

•  You  enter  the  first  house.  The  head  of  the  household  (HoH)  invites  you  and  your  LNS  into 
the  kitchen.  The  rest  of  the  team,  including  your  interpreter,  goes  off  with  another  household 
member  to  search  the  rest  of  the  house.  You  (the  Player)  and  the  LNS  follow  the  HoH  into 
the  kitchen.  You  suspect  he  might  be  an  important  person  in  the  village. 

Here  are  several  possible  outcomes  for  the  encounter: 

•  In  the  kitchen  of  a  home,  the  head  of  household  (HoH)  is  an  older  civilian  man,  very  wary  of 
both  the  player  and  the  LNS.  The  LNS  is  young  and  arrogant. 

•  After  an  initial  greeting,  the  HoH  offers  the  player  fruit,  in  an  act  of  goodwill.  Meanwhile  the 
LNS,  is  wandering  around  the  kitchen,  picking  up  objects  without  asking,  which  begins  to 
anger  the  HoH.  The  player  can  gesture  for  the  LNS  to  stop  doing  that  and/or  to  come  stand 
with  the  player,  which  builds  rapport  with  the  HoH. 

•  The  LNS  and/or  the  player  can  point  to  some  cabinets,  and  ask  the  HoH  to  open  them. 
Depending  on  the  level  of  rapport,  the  HoH  may  initially  balk.  Eventually  the  HoH  will 
reluctantly  open  a  cabinet,  showing  it  is  empty. 

•  The  player  has  the  option  of  gesturing  for  the  HoH  to  calm  down,  including  putting  a  hand  on 
the  HoH’s  shoulder,  in  an  attempt  to  pacify  him. 

•  The  LNS  and/or  the  player  can  ask  for  a  second  cabinet  to  be  opened,  which  reveals  storage 
of  illicit  medicine. 

•  The  LNS  eventually  picks  up  a  piece  of  bread  on  the  table,  to  eat  it.  This  infuriates  the  HoH, 
who  smashes  a  bowl  of  fruit  in  defiance. 

•  In  response  to  that  possible  threat  of  violence,  the  LNS  and/or  the  player  may  raise  their 
rifles  to  a  low-ready.  This  causes  the  HoH  to  become  frightened  and  call  for  his  son,  who 
enters  the  room,  scared  and  angry,  yelling  at  the  LNS  and  player  to  leave.  The  player  can 
gesture  for  the  LNS  and  player  to  leave  the  kitchen,  to  de-escalate  the  situation. 

•  Alternatively,  if  the  player  had  prevented  the  LNS  from  eating  bread,  the  HoH  may  be  less 
upset.  The  son  will  eventually  wander  in,  and  the  HoH  will  introduce  him  to  the  LNS  and 
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player,  at  which  point  the  player  can  gesture  for  LNS  and  player  to  leave  calmly. 

An  alternative  version  of  Breaking  Bread  was  partially  implemented  in  which  the  HoH  is  female 
and  the  LNS  is  submissive,  indicating  that  you  have  found  yourself  in  a  matriarchal  society. 

3.1.4  Sandbox  Demonstration 


Figure  8:  Screenshot  from  the  Sandbox  Scenario 


The  Sandbox  is  an  outgrowth  of  the  earlier  dynamics  testbed  demonstration,  and  reflects  the 
realization  that,  while  scenario-based  training  is  very  important,  it  is  not  the  only  framework  for 
delivering  meaningful  social  experiences.  The  Sandbox  situates  the  learner  in  a  village  square  in 
an  unfamiliar  country.  Two  characters  appear,  one  carrying  a  box  of  fruit.  They  do  not  speak 
English  but  are  friendly,  inquisitive,  accommodating  and  inclusive;  their  objective  is  to  interact 
with  the  learner  and  to  draw  the  learner  into  the  experience.  For  example,  they  may  engage  in  the 
following  behaviors: 

•  Mirror  some  learner  behaviors:  smiles,  bows,  waves  and  head  nods. 

•  React  to  other  learner  behaviors  in  simple  ways;  e.g.,  look  in  the  direction  the  learner  points, 
approach  when  beckoned,  or  startle  if  the  learner  makes  a  loud  vocalization. 

•  Engage  in  simple  exchanges;  e.g.,  at  the  learner’s  request  pass  objects  to  the  learner  and  take 
them  back.  They  can  get  into  mild  disputes  with  each  other  during  these  transactions. 

•  If  the  learner  is  idle,  try  to  get  the  learner’s  attention. 

The  long-term  goals  for  the  Sandbox  were  three-fold:  1)  to  invite  learners  to  interact  with  the 
system,  2)  to  familiarize  learners  with  behaviors  that  the  virtual  characters  can  recognize  and 
react  to,  and  3)  to  serve  as  a  low-key  practice  environment  for  a  variety  of  simple  social 
interactions.  It  also  rewards  fundamental  meta-cultural  skills:  observation,  mimicry,  engagement, 
and  calmness  in  the  face  of  trouble  that  inevitably  arises  in  social  interaction.  The  highly  reactive 
and  playful  nature  of  the  VC  reactions,  along  with  the  easy  pace  of  the  experience  -  nothing 
happens  if  the  learner  takes  a  long  time  to  do  things  except  for  attempts  to  introduce  them  to  new 
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behavior  -  provides  a  very  low  engagement  barrier  as  well  as  the  gratification  of  nearly  instant 
social  feedback. 

The  following  interactions  are  possible  in  the  Sandbox: 

Transactions 

The  box  of  fruit  provides  opportunities  for  offering  and  requesting,  accepting  and  refusing.  Many 
variations  on  this  theme  are  possible: 

•  When  the  Player  extends  his  right  hand  toward  a  VC  holding  some  fruit,  the  VC  will 
approach  and  offer  it  to  the  Player.  In  addition,  the  VC  may  spontaneously  offer  fruit  to  the 
Player,  but  not  every  time. 

•  The  Player  can  take  the  offered  fruit  by  extending  his  hand. 

•  The  Player  can  eat  the  fruit  by  holding  his  hand  to  his  mouth  to  mime  eating.  It  is  possible  to 
eat  the  entire  box  of  fruit,  one  at  a  time.  When  the  box  is  empty,  the  VCs  will  bid  farewell 
and  leave. 

•  The  Player  can  give  the  fruit  back  by  extending  his  hand  toward  a  VC  who  is  not  holding 
anything.  The  VC  will  approach  and  take  the  fruit.  Offering  fruit  to  a  VC  who  is  already 
holding  fruit  will  spark  a  refusal. 

VCs  spontaneously  engage  in  the  giving,  taking  and  eating  fruit,  and  may  encourage  the  Player 
to  participate  in  this  activity. 

Compound  Gestures 

The  Player  can  use  compound  gestures  to  direct  the  VCs. 

When  the  Player  points  at  a  VC,  then  points  to  himself,  this  is  interpreted  as  a  request  to 
approach. 

Show  Gratitude 

The  hand-to-heart  gesture  is  interpreted  as  thank  you,  as  is  a  bow  or  Namaste.  The  VC  will 
mirror  the  gesture,  and  may  introduce  the  hand-to-heart  gesture  when  accepting  fruit  from  the 
Player. 

When  the  Player  smiles,  VCs  will  smile  back. 

Exchange  of  Names 

When  the  Player  looks  at  a  VC,  points  at  himself  and  speaks,  the  VC  interprets  this  as  an 
introduction  and  will  introduce  himself  by  name.  VCs  persist  in  referring  to  the  Player  as 
“Americano”  however.  In  addition,  VCs  may  spontaneously  introduce  themselves  to  the  Player. 
The  VC  with  the  box  of  fruit  is  “Basuki”;  the  other  VC  is  “Yunos”. 

Direct  Attention 

In  general,  pointing  directs  the  attention  of  the  VCs.  The  Player  can  point  at  a  VC,  or  off-screen 
-  up,  down,  left  or  right. 

When  the  Player  says  the  name  of  a  VC,  the  VC  will  respond.  If  the  name  is  accompanied  by  a 
gesture  -  e.g.,  beckon  -  the  named  VC  will  respond. 

Surprise 

When  the  Player  yells  loudly  or  rushes  the  screen,  the  VCs  act  surprised. 
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Agreement 

VCs  mirror  these  behaviors: 

•  Head  nod  -  VC  nods  head 

•  Head  shake  -  VC  shakes  head 

A  Stop,  Stop  That  or  Request  Step  Back  gesture  will  interrupt  whatever  the  VCs  are  currently 
doing  or  to  take  a  step  back  away  from  the  Player. 

Closing 

When  Player  leaves  the  stage  (backs  away  or  steps  sideways  out  of  view),  the  VCs  wave 
goodbye,  turn,  walk  away,  and  go  about  their  business  in  the  square. 

3.1.5  Finding  Farah  Demonstration 


Figure  9:  Screenshot  from  the  Finding  Farah  Scenario 


This  scenario  provides  the  Player  with  a  more  specific  goal  than  just  interacting  with  VCs  in  a 
village  square.  It  incorporates  the  social  interactions  introduced  in  the  Sandbox  demonstration, 
but  takes  place  in  an  urban  setting  and  introduces  a  simple  mission  goal  of  trying  to  solicit  help 
from  a  VC  in  locating  a  person,  Farah.  The  instructional  focus  of  this  scenario  is  to  recognize 
that  time  spent  at  the  beginning  of  an  encounter  building  rapport  and  gaining  trust  will  pay 
dividends  in  accomplishing  the  mission. 

The  Player  is  said  to  be  in  an  unfamiliar  country  on  a  humanitarian  relief  mission:  to  support  the 
local  authorities  to  contain  an  infectious  disease  outbreak.  It  is  important  to  track  down  people 
who  may  have  been  infected  without  their  knowledge  so  they  can  be  treated  before  they  infect 
others.  In  the  Player’s  left  hand  is  a  picture  of  a  woman  named  Farah,  who  needs  to  be  found. 
She  supposedly  lives  nearby. 
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One  VC  recognizes  the  person  in  the  photo,  but  is  reluctant  to  reveal  this.  He  will  show  subtle 
signs  of  recognition,  and  some  unease,  but  that’s  all.  Repeated  attempts  by  the  Player,  prior  to 
establishing  some  kind  of  rapport,  will  produce  reactions  of  increasing  annoyance,  leading  to  the 
VCs  breaking  off  the  encounter  altogether. 

Rapport  building  entails  offering  greetings,  introducing  oneself,  taking  and  eating  the  fruit  that  is 
proffered  by  the  VCs,  asking  for  more  fruit,  or  showing  interest  in  the  surroundings,  and,  above 
all,  being  patient. 

A  second  VC  is  much  more  suspicious,  and  will  actively  discourage  the  other  from  helping  in 
any  way.  The  Learner  may  choose  to  assert  himself  in  resolving  or  overriding  this  conflict 
between  the  VCs,  which  can  lead  to  the  knowledgeable  VC  grudgingly  complying  with  the 
request  to  help  find  the  person  in  the  photo.  A  better  outcome  occurs  when  the  Player  invests 
effort  initially  in  building  rapport,  and  the  VC  willingly  complies  and  offers  to  take  the  Learner 
to  meet  the  person.  A  worse  outcome  occurs  when  the  Players  focuses  exclusively  on  the 
question  of  identity,  and  the  VCs  become  annoyed  and  leave  without  revealing  any  information. 

The  three  possible  results  represent  natural  outcomes  based  on  the  learner’s  competence  as  a 
good  stranger,  they  follow  from  the  SSIM  social  science  results  and  can  serve  as  the  context  for 
pedagogical  interactions  between  the  system  and  the  learner  (coaching  and  AAR)  that  will  help 
the  learner  succeed  and  understand  the  consequences  of  his  or  her  actions.  The  experiences 
produced  by  the  system  constitute  a  kind  of  social  puzzle  that  could  be  a  natural  part  of 
intelligence  gathering,  disaster  relief  or  other  military  missions. 

The  following  interactions  are  possible  in  Finding  Farah: 

•  On  a  quiet  morning  in  a  city  square,  the  player  encounters  two  strangers  -  two  civilian  men 
who  are  friends  with  each  other:  Basuki,  who  is  carrying  a  box  of  fruit,  and  Yunos.  They 
notice  the  player,  approach  and  greet  the  player,  and  attempt  to  build  rapport  by  using 
gestures,  while  speaking  in  their  native  tongue. 

•  Yunos,  somewhat  friendly,  tends  to  try  to  get  Basuki  to  offer  the  player  some  fruit;  Basuki  is 
wary  and  resists.  Yunos  may  eventually  take  some  fruit  from  Basuki  and  offer  it  to  the 
player,  who  can  take  it  and  eat  it,  which  builds  rapport. 

•  If  the  player  tries  to  show  them  and  ask  about  the  photo  before  building  some  rapport,  they 
are  hesitant,  and  Basuki  commands  Yunos  not  to  answer.  If  the  player  badgers  them  about 
the  photo,  they  will  become  annoyed  and  walk  away. 

•  If  the  player  asks  about  the  photo  after  building  rapport,  they  are  more  likely  to  answer, 
pointing  off-screen  to  where  Farah  is,  and  inviting  the  player  to  follow  them  to  her. 

•  If  the  player  asserts  authority  by  commanding  the  resistant  Basuki  to  step  back,  Yunos  may 
angrily  give  you  information  about  Farah,  regardless  of  rapport,  and  then  leave  without 
inviting  the  player  to  follow  them. 

3.2  Deployments 

In  order  to  illustrate  the  capabilities  of  the  SSIM  simulator  and  its  potential  for  training,  as  well 
as  to  disseminate  the  lessons  learned  during  the  development  of  the  SSIM  simulator,  the 
simulator  prototype  system  has  been  deployed  three  times  to  DoD  organizations,  which  can 
evaluate  its  potential  and  communicate  that  potential  to  a  wider  circle  of  DoD  entities  that  would 
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benefit  from  training  based  on  SSIM  technology.  BBN,  working  under  the  direction  of  DARPA, 
deployed  prototypes  at  the  following  three  organizations. 

3.2.1  United  States  Air  Force  Academy 

In  November  2015  we  deployed  a  version  of  the  SSIM  simulator  to  the  Warfighter  Effectiveness 
Research  Center  at  U.S.  Air  Force  Academy  in  Colorado  Springs.  They  were  eager  to  have  their 
cadets  evaluate  the  experiences  produced  by  the  system  and  to  give  feedback  on  its  operation. 
BBN  suggested  that  it  might  be  appropriate  to  measure  social  engagement  (as  opposed  to 
ordinary  engagement  with  a  software  system)  between  the  human  learner  and  the  virtual 
characters.  Air  Force  Academy  cadets  designed  and  executed  evaluation  experiments  as  part  of 
their  capstone  activities  during  the  2015  spring  semester.  They  presented  the  results  of  their 
work  to  DARPA  SSIM  leadership  on  May  6th,  2015.  Among  other  findings  they  reported  that 
more  than  60%  of  the  test  subjects  felt  that  the  initial  prototype  was  useful;  they  wished  it  had  a 
more  defined  mission  and  gave  feedback  (both  capabilities  have  been  addressed  in  the  final 
prototype).  The  Air  Force  Academy  intends  to  continue  evaluating  the  final  version  of  the  SSIM 
prototype  system,  which  they  received  in  May  of  2015. 

3.2.2  US  Army  Training  Brain  Operations  Center 

In  March  2015  we  delivered  a  version  of  the  SSIM  simulator  to  the  U.S.  Army  Training  Brain 
Operations  Center  (TBOC)  in  Newport  News  VA.  We  trained  TBOC  personnel  on  the  system’s 
operation  and  purpose  and  also  conducted  detailed  technical  overviews  of  the  system  with 
TBOC’s  programming  staff.  They  intended  to  show  the  system  to  various  potential  users  (from 
DoD  and  the  Intelligence  Community)  of  the  technology  in  the  SSIM  system. 

3.2.3  US  Marine  Corps  Center  for  Advanced  Operational  Culture  and  Learning 

In  May  2015  we  delivered  the  final  version  of  the  prototype  to  the  U.S.  Marine  Center  for 
Advanced  Operational  Culture  Beaming  (CAOCF)  at  TECOM  in  Quantico  VA.  CAOCF 
personnel  were  trained  to  operate  the  system.  CAOCF  intends  to  evaluate  SSIM  technology  for 
its  utility  in  Marine  education  and  training.  Their  evaluation  is  to  take  place  during  the  following 
year. 

3.3  Publications  and  Presentations 

The  following  publications  and  presentations  were  conducted  by  members  of  the  IMMERSE 
team  as  part  of  the  SSIM  effort. 

Ferguson,  W.,  Roberts,  B.,  Diller,  D.,  Shapiro,  D.  and  Mateas,  M.  Teaching  Cross  Cultural 
Social  Competence  in  a  Dynamic,  Synthetic  Environment.  Proceedings  of  the 
Interservice/Industry  Training,  Simulation,  and  Education  Conference,  Paper  No.  14289, 
Orlando,  FF,  2014. 

Gratch,  J.,  Mateas,  M.,  Ross,  T.,  Johnson,  W.F.,  Ferguson,  W.,  &  Butler,  W.  (2013).  Artificial 
Intelligence  for  Social  Interaction  Simulation.  Special  Panel  Session  of  the 
Interservice/Industry  Training,  Simulation,  and  Education  Conference,  Orlando,  FF,  2013. 

Klein,  G.,  Klein,  H.  A.,  Fande,  B.,  Borders,  J.,  &  Whitaker,  J.C.  (2014).  The  Good  Stranger 
frame  for  police  and  military  activities.  Proceedings  of  the  Human  Factors  and  Ergonomics 
Conference,  Sept  2014. 
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Klein,  H.  A.  and  Borders,  J.  (2015).  Good  Stranger  Diagnostic  Tool:  Measuring  capacitates  and 
limitations  to  inform  training.  Accepted  for  presentation  at  the  12th  International  Naturalistic 
Decision  Making  Conference,  McLean,  VA. 

Klein,  G.,  Klein,  H.  A.,  Lande,  B.,  Borders,  J.,  Whitacre,  J.  C.  (2015).  Police  and  military  as 
Good  Strangers.  Journal  of  Occupational  and  Organizational  Psychology  -  Special  selection 
paper  (2015). 

Mateas,  M.  and  Ferguson,  W.  (2014).  Social  Legos:  Toward  Modular,  Reusable  Social  Behavior. 
Proceedings  of  the  2014  Game  Developers  Conference  (GDC),  San  Francisco,  CA. 

Shapiro,  D.,  McCoy,  J.,  Grow,  A.,  Samuel,  B.,  Stem,  A.,  Swanson,  R.,  Treanor,  M.,  and  Mateas, 
M.,  Creating  Playable  Social  Experiences  through  Whole -body  Interaction  with  Virtual 
Characters  (2013).  Proceedings  of  the  Ninth  AAAI  Conference  on  Artificial  Intelligence  and 
Interactive  Digital  Entertainment  (AI1DE13),  Boston,  MA. 

Shapiro,  D.,  Tanenbaum,  K.,  McCoy,  J.,  LeBron,  L.,  Reynolds,  C.,  Stem,  A.,  Mateas,  M., 
Ferguson,  B.,  Diller,  D.,  Moffitt,  K.,  Coon,  W.,  and  Roberts,  B.,  Composing  Social 
Interactions  via  Social  Games,  International  Conference  on  Autonomous  Agents  and 
MultiAgent  Systems  (AAMAS)  2015,  Istanbul,  Turkey. 
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4.  Simulator  System  Description 

The  following  section  describes  the  Social  Simulator  System’s  primary  components  and  the 
architecture  used  to  integrate  these  components. 

4.1  System  Architecture 

Figure  10  shows  the  top-level  system  architecture.  The  learner,  shown  at  the  far  left  in  Figure  10, 
interacts  with  the  system  in  the  same  way  the  learner  would  interact  with  real  people:  through 
speech,  gestures,  comportment,  and  other  forms  of  nonverbal  communication.  Learner  input  into 
the  system  is  handled  by  the  learner- sensing  system,  MiBA  (Multimodal  Integrated  Behavior 
Analysis),  which  takes  body  and  voice  input  from  a  Kinect  2  sensor  and  microphone;  analyzes 
the  learner’s  gaze,  facial  expressions,  body  pose,  gestures,  speech,  speech  vocalics,  and 
multimodal  affect;  and  outputs  this  analysis  as  a  rich  physical  description  of  learner  behavior. 
These  signals  are  passed  along  to  the  Social  Pragmatics  (SP)  module,  which  performs  the  last 
stage  of  input  interpretation  in  order  to  provide  data  in  a  form  comprehensible  to  the  Social 
Simulation  Engine  and  Performance  AI  components.  The  Social  Simulation  Engine  creates  a  rich 
social  environment  for  learning  and  models  the  high  and  intermediate  level  activities  of  the 
learner  and  the  VCs,  while  its  tightly  integrated  companion  the  Performance  AI  module 
determines  the  moment-by-moment  behaviors  of  the  VCs  and  the  coach.  Performance  AI  drives 
all  of  the  system  output  seen  and  heard  by  the  learner,  which  is  ultimately  presented  by  the 
Virtual  World  Simulator:  the  game  engine  rendering  the  VCs  and  the  simulated  environment. 

The  Pedagogical  Director  (PD)  is  responsible  for  assessing  the  learner  and  influencing  the 
learner’s  experience  to  provide  an  effective,  individualized  and  varied  training  experience. 
Coaching,  which  is  very  much  a  social  interaction  itself,  is  implemented  using  the  mechanisms 
of  the  Social  Simulation  Engine  and  the  Performance  AI.  Each  of  these  components  is  described 
in  more  detail  in  the  sections  below. 
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Figure  10:  System  Architecture 

4.2  System  Components 

4.2.1  MiBA 

The  Multimodal  Integrated  Behavior  Analysis  (MiBA)  sensing  application  was  developed  by 
SRI  International’s  Princeton  research  laboratory,  and  serves  as  the  eyes  and  ears  for  the  Social 
Simulator.  It  runs  on  a  separate  computer  and  takes  input  from  a  Kinect  2  sensor  and  separate 
microphone  and  analyses  the  learner’s  gaze,  facial  expressions,  body  pose,  gestures,  speech, 
speech  vocalics,  and  multimodal  affect  and  converts  this  input  into  a  rich  set  of  possible  social 
interpretations.  These  signals  are  passed  along  to  the  Social  Pragmatics  component,  which 
performs  the  last  stage  of  input  interpretation. 

4.2.2  Social  Pragmatics 

The  Social  Pragmatics  component  serves  dual,  complementary  roles  as  a  translator,  reading  the 
streams  of  raw  data  that  MiBA  provides  and  distilling  them  into  events  that  the  Social 
Simulation  Engine  can  understand;  and  as  a  careful  filter,  implementing  pragmatic,  empirically- 
designed  rules  that  govern  exactly  which  MiBA  signals  (at  what  strength  and  for  what  duration, 
with  or  without  other  signals)  should  generate  events  for  the  Social  Simulation  (See  Section 
4.2.3). 

In  a  sense,  Social  Pragmatics  acts  as  a  kind  of  bridge  to  a  utopian  world:  if  some  perfect  MiBA 
existed,  that  behaved  exactly  as  we  might  hope  (an  impossibly  tall  order  given  the  messy  nature 
of  the  world  it’s  sensing  and  the  constraints  imposed  by  the  hardware  on  which  it  relies)  there 
would  be  no  role  for  Social  Pragmatics;  MiBA  would  simply  connect  directly  to  the  Social 
Simulation  and  issue  events  without  mediation.  One  can  see  this  play  out  in  the  co-evolution  of 
MiBA  and  Social  Pragmatics  over  time:  as  MiBA  improved  over  the  course  of  the  project  and  its 
output  became  more  consistent  and  reliable,  the  complexity  and  role  of  Social  Pragmatics 
diminished. 

We  should  note  that  while  it  plays  an  important  role  in  the  system,  Social  Pragmatics  is  limited 
by  the  fact  that  it  makes  no  use  of  contextual  semantic  information;  its  activity  is  governed 
entirely  by  static  rules  (implemented  in  Drools)  that  are  not  informed  by  the  current  state  of  the 
Social  Simulation  Engine.  Additional  signal  processing  -  within  the  ABL  AI  code  -  handles 
disambiguation  of  Social  Pragmatics’  output  based  on  the  dynamic  social  state  of  the  running 
system.  (Social  Pragmatics  does  allow  for  -  static  -  differences  between  agents;  for  example, 
some  agents  might  understand  more  English  than  others.) 

4.2.3  MiBA  Signal  Visualization 

In  the  course  of  analyzing  signals  output  from  the  MiBA  system  (for  the  purpose  of  adjusting  the 
Social  Pragmatics  rules)  it  became  clear  that  visualization  of  the  data  stream  is  critical  given  the 
volume  and  nature  of  the  data.  We  developed  a  Javascript  graphing  tool  built  on  amCharts  to 
serve  that  purpose.  Figure  1 1  shows  what  the  visualization  looks  like  in  a  static  screenshot,  but 
its  real  value  comes  from  its  flexibility  and  interactivity:  data  from  an  entire  session  can  be 
imported  in  seconds;  specific  signals  can  be  toggled;  the  selection  region  and  zoom  is  easy  to 
adjust  using  the  slider  and  handles  presenting  the  timeline  at  the  top;  and  mousing  over  any  point 
on  any  of  the  signal  traces  shows  the  precise  value(s)  at  that  point. 
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Figure  11:  MiBA  Signal  Visualizer 

4.2.4  Social  Simulation  Engine  and  Performance  Al 

These  two  companion  components  are  implemented  primarily  in  ABL,  an  agent-oriented 
language  that  compiles  to  Java.  They  accept  social  signals  and  the  state  of  the  virtual  world,  and 
reason  over  possible  actions  for  virtual  characters  to  take.  When  faced  with  a  pedagogically 
significant  choice,  some  configurations  of  the  Social  Simulation  Engine  pass  action  options  to 
the  Pedagogical  Director  (implemented  by  SoarTech  in  Java  -  see  Section  4.2.7),  which 
communicates  its  choice  back  to  the  Social  Simulation  Engine. 

The  choices  made  by  these  two  components  bottom  out  into  animation  acts,  which  the 
Performance  AI  module  sends  to  the  virtual  world  simulator. 

For  further  discussion  about  the  activity  of  this  component,  see  Section  5:  Expressive  Artificial 
Intelligence  for  Social  Immersion. 

4.2.5  Virtual  World  Simulator 

A  custom  mechanism  built  with  Unity  -  a  highly  flexible,  general-purpose  3D  game  engine  - 
renders  all  of  the  audio  and  video  that  IMMERSE  end  users  actually  experience. 

A  socket-based  bridge  connects  Unity  to  the  Performance  AI  module  and  transacts  using  sets  of 
name-value  pairs  that  describe  commands  going  into  Unity  and  events  coming  out  of  it.  The  API 
that  this  bridge  implements  (in  C#  on  the  Unity  side)  provides  significant  power  and  flexibility  to 
the  Performance  AI  code  that  drives  it.  Highlights  include  support  for  layered,  body-part- 
specific  animation  (including  a  multi-layered  facial  animation  mechanism),  audio  playback, 
dynamic  VC  gaze  tracking,  and  VC  locomotion  and  object  manipulation. 

Inverse  kinematics  (IK)  is  used  for  several  features  in  the  virtual  world  simulation:  controlling 
character  gaze,  pointing,  and  interaction  with  objects.  Character  gaze  uses  Unity’s  built-in  IK 
system,  with  a  fairly  broad  support  framework  built  on  top  of  it.  Unity’s  IK  system  positions 
characters  correctly  for  looking  at  particular  points  in  a  given  moment,  while  the  support 
framework  uses  this  to  allow  characters  to  smoothly  transition  between  looking  at  different 
things,  focus  on  different  targets  with  their  head  and  eyes,  and  so  forth.  Final  IK,  a  Unity  plugin, 
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is  used  to  handle  pointing,  and  object  interaction.  When  a  character  is  properly  rigged  with  Final 
IK  components,  pointing  can  be  controlled  (and  transitioned  in  and  out)  in  much  the  same  way  as 
with  the  built  in  Unity  IK  system  -  but  with  a  more  advanced,  more  realistic  constraint  solver. 

Object  interaction  is  a  little  more  complex.  Each  object  that  can  be  interacted  with  (picked  up, 
opened,  etc.)  has  a  component  describing  the  correct  body  position  and  animation  curves 
required  for  the  interaction.  For  example,  the  fruit  in  the  fruit  box  have  a  model  of  the  proper 
hand  position  for  picking  them  up.  This  combines  with  a  character- side  component  that  smooths 
out  animations,  handles  updating  of  inventories,  and  so  forth,  to  allow  each  VC  to  interact  with 
the  environment  in  a  number  of  different  ways. 

4.2.6  Models  and  Assets 

BBN  subcontractor  Fire  Hose  Games  produced  the  visual  assets  in  IMMERSE. 

BBN,  UCSC,  and  Fire  Hose  worked  together  in  a  phased  and  iterative  process  to  design  and 
develop  the  virtual  characters,  and  the  settings  in  which  the  various  scenarios  occur.  This  began 
at  the  start  of  the  program,  with  extensive  and  intensive  discussion  and  prototyping  to  determine 
a  level  of  realism  appropriate  to  the  task.  On  one  hand,  we  did  not  want  to  aim  for  photorealism, 
both  because  of  the  Uncanny  Valley  effect  (in  which  very-nearly-but-not-quite-real-looking 
characters  produce  a  confused,  disturbed,  or  even  disgusted  state  in  viewers)  and  because  fidelity 
issues  would  make  some  of  the  more  subtle  virtual  character  expressions  difficult  for  a  user  to 
perceive.  On  the  other  hand,  we  did  not  want  to  make  a  cartoon  world;  we  knew  our  end  users 
would  require  some  realistic  aspects  if  this  were  some  day  to  become  a  full-fledged  and  effective 
training  system.  Ultimately,  we  embraced  a  balanced  approach  that  incorporated  enough 
realistic  elements  that  the  user  could  accept  the  world  as  an  alternate  reality  (not  a  cartoon  world) 
but  enough  exaggerated  and  simplified  elements  that  we  avoided  the  uncanny  valley,  and  the 
expressions  of  the  virtual  characters  were  immediately  clear. 

When  designing  the  sets,  we  made  extensive  use  of  real-world  photographs  and  other  source 
material;  and  produced  (and  iterated  on)  concept  art,  rough  untextured  geometry,  and  rough 
texturing  on  the  way  to  producing  the  final  scenes. 

All  character  models  and  animations  made  use  of  the  skinned  animation  mechanism  provided  by 
Unity  (“Mecanim”)  for  everything  including  facial  animation  and  lip  synch  but  excluding  some 
advanced  Inverse  Kinematics  functionality,  which  is  provided  by  a  third-party  library  (See 
Section  4.2.5,  above). 

Because  we  opted  to  provide  the  Performance  AI  module  with  maximum  control  over  the 
animation  system,  we  made  minimal  use  of  the  Unity  animation  state  machines  that  a  game 
might  typically  use  to  control  animation  transitions.  Instead,  we  established  a  database  of  all 
animation  assets  (implemented  as  an  Excel  spreadsheet)  that  specifies  how  each  animation  is  to 
be  used  in  terms  of  body  parts  and  layering,  that  we  then  use  to  generate  automatically  the  Java 
code  describing  the  animations  (used  by  ABL  code)  and  the  (rudimentary)  state  machines  used 
by  Unity  to  understand  the  animation  assets  and  their  usage. 

4.2.7  Pedagogical  Director 

The  Social  Simulation  Engine  is  integrated  with  the  Pedagogic  Director  (PD),  developed  by 
SOAR  Technologies,  another  TA-2  performer.  The  PD  has  an  important  role  in  tracking  learner 
performance  over  time  against  a  collection  of  learning  objectives.  And,  in  any  given  scenario 
run-through,  it  can  set  initial  conditions  and  moderate  the  activities  of  VCs  and  the  coach,  though 
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a  pedagogical  policy  that  attempts  to  balance  three  competing  goals:  social  scores  (the  social 
simulation  engine’s  prioritization  of  available  social  moves),  learning  objectives  (for  example, 
focusing  on  unmet  learning  objectives),  and  variability  (exposing  the  learner  to  unfamiliar 
circumstances). 

As  described  in  detail  in  Section  5, Expressive  Artificial  Intelligence  for  Social  Immersion,  the 
social  simulation  engine  repeatedly  generates  a  ranked  list  of  social  moves  and  their  associated 
Social  Interaction  Unit  (SIUs).  Each  list  of  proposed  SIUs  is  sent  to  the  PD  for  consideration. 
Each  SUI  option  includes  metadata  describing  the  initiator  and  responder  of  the  activity,  and  a 
set  of  learning  objectives  impacted  by  the  SIU.  The  learning  objectives  include  immediate  and 
future  learning  objectives.  Future  learning  objectives  allow  the  PD  to  factor  the  anticipated 
consequences  of  critical  junctures  into  its  decision-making.  SIUs  generated  by  the  social  coach  - 
proposing  instructional  interventions  -  were  likewise  passed  along  to  the  PD  for  adjudication. 
The  learning  objectives  were  drawn  from  the  findings  of  the  TA-1  and  TA-1B  program 
activities. 

For  additional  discussion  regarding  the  Pedagogic  Director,  and  coaching  and  pedagogy 
generally,  see  Section  6,  Social  Pedagogy. 

4.3  Infrastructure 

4.3.1  System  Hardware 

Typical  IMMERSE  installations  includes  the  following  hardware  components: 

•  70”  Television  (Sharp  LC-70LE650U) 

•  Television  Stand  (TVSTN80) 

•  Kinect  for  Windows  v2 

•  Microphone  Assembly  (Shure  BLX88,  BLX1,  PGA31) 

•  MiBA  Workstation  (AlienWare  Aurora  R4) 

•  Simulator  Workstation  (AlienWare  Aurora  R4) 

•  Network  Switch  (Linksys  SE2800) 

The  two  workstations  communicate  over  a  local  network  that  includes  nothing  more  than  those 
two  machines  (no  external  network  connection  is  required). 

4.3.2  Software  Repositories 

At  the  highest  level  of  organization,  the  IMMERSE  software  makes  use  of  three  different 
software  repositories:  an  SVN  repository  (“live-build”)  containing  the  latest  stable  interoperating 
binaries  from  all  performers  (as  well  as  various  test  branches)  and  two  Git  repositories  (“ssim” 
and  “ssim-unity”)  for  the  main  system  framework  (including  all  ABL  AI  code)  and  the  Unity 
project,  respectively.  The  remainder  of  this  section  focuses  on  the  system  framework  contained 
in  the  “ssim”  repository. 

4.3.3  Processes  and  Interprocess  Communications 

As  shown  in  the  High-Level  Data  Flow  Diagram  (Figure  12),  IMMERSE  is  composed  of  four 
processes.  (The  three  external  to  the  core  framework  are  shown  in  red.) 

The  Social  Simulation  and  Unity  processes  are  generally  run  on  the  same  workstation  due  to  the 
tightly  coupled  and  high-velocity  nature  of  their  transactions,  which  are  structured  as  sets  of 
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simple  name-value  pairs.  MiBA  is  run  on  a  dedicated  workstation  (given  its  processing 
requirements)  and  is  controlled  via  a  small  Remote  Launcher  application  (designed  to  run  at 
startup)  that  exposes  MiBA  as  a  service,  so  that  users  never  have  to  do  anything  with  the  MiBA 
workstation  except  start  it  up  and  shut  it  down. 
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Figure  12:  High-Level  Data  Flow  Diagram 

Transactions  with  MiBA  are  encoded  as  XML,  and  conducted  over  sockets.  Because  much  of 
the  streamed  data  MiBA  provides  is  effectively  binary,  we  require  careful  use  of  explicit 
schemas  (encoded  as  XSD)  -  and,  we  use  these  schemas  to  generate  the  Java  code  representing 
the  structure  of  the  MiBA  streams.  (MiBA  also  exposes  a  REpresentational  State  Transfer 
(REST)  interface  that  the  system  uses  for  high-level  state  management  during  initialization  and 
calibration.) 

Although  the  Pedagogic  Director  initially  used  an  XML-over- sockets  interface,  it  eventually 
became  a  service,  with  which  the  Social  Simulation  communicates  over  a  REST  interface  when 
the  PD  is  in  use.  (The  PD  is  designed  to  run  on  a  dedicated  server  that  is  shared  across  multiple 
applications.) 

4.3.4  System  Framework  and  Execution 

The  IMMERSE  software  framework  is  implemented  in  Java,  using  the  Spring  Framework  in  a 
lightweight  way  to  organize  the  various  software  objects  (“beans”,  in  the  Spring  Framework 
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terminology  -  in  Java  they  are  simply  classes,  with  no  special  designation  except  that  they  are 
referenced  in  Spring  configuration  files  -  see  AblBridge/resource/spring)  into  various  launch 
configurations.  Optional  configuration  elements  include  the  Unity  process  launcher  (which  is 
disabled  in  some  configurations  to  allow  for  the  use  of  the  Unity  Editor,  which  launches  the 
Unity  process  independently)  and  the  MiBA  GUI  Server  element,  which  provides  a  GUI-based 
implementation  of  the  MiBA  system,  for  use  in  development  environments  that  do  not  have  the 
full  MiBA  system  installed.  Another  configuration  variation  involves  different  flavors  of  the 
Pedagogic  Director  (REST-based,  socket-based,  trivial,  none).  All  launch  configurations  start 
with  the  SpringMain  class,  which  loads  low-level  properties  files  ([config,  site,  user]  .properties) 
and  then  loads  one  or  more  XML  files  describing  the  set  of  beans  to  include  in  the  current 
configuration  (and  how  they  are  connected  to  each  other). 

The  Spring  Bean  Dependency  Graph  (Figure  13)  shows  the  core  set  of  beans  that  are  included  in 
all  configurations.  Highlights  include  the  three  client  beans  that  read  data  from  MiBA  and 
implement  Social  Pragmatics;  the  simulation  bean  that  transacts  with  the  Unity  process;  the 
localizer  bean  that  monitors  all  agents  and  performs  spatial  reasoning;  and  the  ablManager  bean 
that  initializes  and  provides  an  interface  to  the  ABL  runtime. 


Figure  13:  Spring  Bean  Dependency  Graph 
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5.  Expressive  Artificial  Intelligence  for  Social  Immersion 

This  section  describes  the  technology  we  have  developed  for  supporting  unscripted  social 
interaction  with  virtual  characters.  Although  people  generally  treat  social  competence  as  a 
basic/implicit  skill,  it  is  difficult  to  achieve  with  virtual  characters  for  several  reasons.  First, 
social  phenomena  are  inherently  complex  and  require  sophisticated  mechanisms  to  express  in 
automated  form.  In  addition,  there  is  a  gap  between  computational  models  and  social  science 
theory,  as  no  one  has  attempted  to  transform  the  types  of  behavioral  analyses  available  from  the 
social  sciences  into  generative  models  that  can  drive  synthetic  characters.  Our  approach  attempts 
to  bridge  this  gap  by  providing  virtual  characters  with  a  social  agenda.  In  particular,  we  take 
inspiration  from  Berne  (1966)  and  Goffman  (1974),  and  enable  non-scripted  interactions  with 
virtual  characters  by  pursuing  the  metaphor  of  social  games. 

In  more  detail,  we  define  a  social  game  as  a  family  of  practices  surrounding  particular  elements 
of  social  state,  and  associate  social  motivations,  moves,  and  preference  logic  with  each  game, 
together  with  a  body  of  executable  behaviors  that  implement  those  moves.  Our  architecture  for 
utilizing  social  games  models  intent  formation  by  selecting  character  motivations  appropriate  to 
the  situation,  and  resolving  them  through  social  moves  into  behavior.  It  selects  character 
behaviors  in  response  to  player  input  through  similar  preference  reasoning.  The  underlying 
behaviors  execute  in  a  virtual  world  that  supports  real-time,  whole-body  player  interaction. 

The  behaviors  that  drive  character  action  constitute  a  major  body  of  work.  At  an  abstract  level, 
they  provide  the  standard  functions  of  a  reactive  computational  agent  that  senses,  thinks,  acts  on 
and  reacts  to  the  world.  However,  our  context  imposes  two  additional  constraints:  1)  the 
behaviors  must  represent  social  functions,  and  2)  they  must  generate  expressive,  believable 
performances  if  the  player  is  to  engage  socially  with  the  characters  at  all.  These  constraints 
structure  our  performance  system.  The  characters  sense  the  world  by  interpreting  player  input  in 
social  terms,  they  represent  plans  as  compositions  of  short  social  interactions  that  respond  to 
player  input,  and  we  have  put  a  great  deal  of  effort  into  the  small  behavioral  nuances  that 
communicate  character  affect,  attention,  and  social  intent.  More  broadly,  our  goal  of  supporting 
non-scripted  social  interaction  has  led  us  to  a  generative  model,  where  the  characters  freshly 
choose  behavior  in  response  to  each  successive  player  signal  and  their  own  internal  motivations, 
rather  than  select  actions  out  of  a  tightly  controlled  script.  As  a  result,  each  interaction  with  the 
player  is  different,  and  each  scenario  has  an  emergent  flow  (within  the  sphere  of  interactions 
supported  by  social  games). 

The  following  material  describes  our  approach  to  social  simulation.  We  divide  that  discussion 
into  sections  on  Social  Games,  covering  intent  formation,  social  response,  and  composition  of 
social  games,  and  Believable  Performance,  which  examines  social  perception,  expressing  social 
interaction,  performance  guidance,  and  the  mechanism  for  interleaving  performances.  The 
underlying  implementation  language  (ABL)  supports  hierarchical,  parallel,  and  reactive  agent 
programs,  using  the  metaphor  of  dynamic  behavior  trees.  It  is  described  in  Mateas  and  Stern 
(2002). 

5.1  Social  Games 

A  social  game  is  a  family  of  practices  surrounding  social  state.  We  have  defined  three  social 
games,  Alliance,  Authority,  and  Threat,  and  have  employed  them  in  multiple  scenarios.  The 
Alliance  game  deals  with  forging  and  damaging  relationships,  while  Authority  concerns  power 
dynamics  and  ownership  over  objects  and  space.  Threat  deals  with  the  management  of  physical 
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force  and  danger.  Each  game  monitors  certain  elements  of  state,  and  contains  representations  of 
relevant  social  motivations  and  abstract  social  actions  called  moves,  together  with  rules  for 
prioritizing  them. 

Figure  14  illustrates  the  architecture  for  executing  social  games.  It  consists  of  a  memory  (for 
holding  background  knowledge,  observations,  and  inferred  beliefs),  an  intention  formation 
process  for  selecting  motivations  and  moves  for  characters  across  games,  and  a  performance 
mechanism  for  executing  and  interleaving  the  associated  behaviors.  We  discuss  social  game 
content  (state,  motivations,  moves,  and  ranking  rules)  and  the  process  of  utilizing  games  to  select 
social  moves,  below. 
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Figure  14:  The  Architecture  for  Social  Games 


We  represent  the  social  game  state  via  binary  predicates  or  continuously  valued  relations.  For 
example,  the  Alliance  game  monitors  a  degree  of  alliance  (a  scalar-valued  directed  relation 
between  two  VCs,  or  the  VC  and  the  player),  and  a  binary  predicate  encoding  whether  those 
agents  are  allies.  Each  social  game  also  contains  a  characteristic  set  of  motivations  to  create 
(increase)  or  remove  (decrease)  that  state,  as  well  as  to  express  its  value  through  agent  actions  in 
the  virtual  world.  The  Alliance  game  supports  AllianceUp,  AllianceDown,  and  AllianceExpress 
relative  to  the  degree  of  alliance,  and  AlliesMake,  AlliesBreak,  and  AlliesExpress  relative  to  the 
allies  distinction.  The  moves  within  a  social  game  represent  abstract  actions  for  pursuing 
motivations.  In  general,  there  are  multiple  moves  for  each  motivation.  For  example,  Greet  and 
GiveAGift  address  AllianceUp,  while  Backup  Ally  and  Friendly  Goodbye  address 
AllianceExpres  s . 

As  a  second  example,  the  Authority  game  governs  interactions  related  to  power  dynamics  and 
ownership  over  objects  and  spaces.  The  underlying  state  consists  of  AuthorityOver(X,Y,  val) 
relations,  and  permission  levels  for  access  to  objects  ranging  from  “no  access”  (0)  to  “can 
consume/destroy”  (5).  The  motivations  are  AuthorityUp,  AuthorityDown,  Authority  Express. 
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The  Authority  game  contains  a  wide  variety  of  moves  for  the  house  search  scenario.  These 
include  MessWithOthersStuff,  OpenYourCabinets,  EatOthersStuff,  HoardMyStuff,  Outburst, 
PutMyStuffDown,  GiveMyStuffBack,  DirtyLook,  and  DerisiveChuckle. 

5.1.1  Intention  Formation 

Each  social  game  contains  a  set  of  preference  rules  for  ranking  motivations  and  moves.  For 
example,  the  following  rules  simultaneously  nominate  and  rank  motivations: 

1)  Alliance(X,  Y,  <50)  AllianceUp(X,Y)  +  2 

2)  Alliance(X,  Y,  >50)  A  Alliance(Y,  Z,  >50)  A  Alliance(X,  Z,  <50)  -  AllianceUp(X,Z)  +  3 

The  first  rule  acts  to  elevate  the  importance  (called  volition)  of  increasing  the  degree  of  alliance 
between  two  characters  whose  bond  is  suffering.  The  second  motivates  a  character  to  build 
networks  of  alliances.  X  observes  that  he  is  friendly  with  Y  but  not  Z  and  Y  is  friendly  to  Z,  so 
X  reaches  out  to  Z. 

Similarly,  the  Authority  game  contains  these  two  rules: 

1)  Arrogant(X)  A  Authority(Y,X,<50  A  Authority(XY,<80)  -»•  AuthorityUp(XY)  +  6 

2)  Permission(X,  Obj,  Owns)  A  Permission(Y,  Obj,  LookAt)  A  Touch(Y,  Obj)  ->• 
AuthorityUp(XY)  +  8 

The  first  expresses  an  inclination  for  one  character  to  dominate  another,  while  the  second 
expresses  a  desire  to  reassert  authority  when  a  character  exceeds  its  allowed  permission  to 
manipulate  an  object. 

An  analogous  set  of  preference  rules  govern  moves.  For  example: 

1)  AllianceUp(XY)  AGiftingAppropriate(XY)  A  Object(tag==“gift”,  Obj)  ->■  GiveAGift(X,  Y, 

Obj)  +  5 

2)  AuthorityUp(X,  Y)  A  Permission(Y,  Obj,  Owns)  A  Permission(X,  Obj,  <Touch)  -»• 
ViolateObjNorm(X,  Obj)  +  7 

These  rules  simultaneously  nominate  and  rank  moves.  The  first  suggests  X  give  an  appropriate 
object  to  Y  as  means  of  increasing  the  degree  of  alliance  between  those  two  characters.  The 
second  rule  incites  X  to  deliberately  exceed  permission  levels  with  respect  to  Y’s  belongings  in  a 
bid  to  increase  authority. 

In  general,  social  games  contain  a  large  number  of  rules  for  ranking  motivations  and  moves,  and 
these  rules  can  depend  upon  accumulated  knowledge,  character  status,  and  current  environmental 
state.  For  example,  GiftingAppropriate(X,  Y)  only  becomes  true  after  characters  have  greeted 
one  another,  and  the  environment  must  contain  objects  tagged  as  giftable  for  the  GiveAGift  rule 
to  apply.  Similarly,  only  characters  flagged  as  Arrogant  seek  to  dominate  others  within  the 
Authority  game. 

Volition  values  add  across  rules.  For  example,  in  the  Breaking  Bread  scenario,  the  player  and 
Focal  National  Soldier  (FNS)  begin  with  an  Alliance,  so  if  the  player  and  Head  of  Household 
(HoH)  become  allies,  the  motivation  rules  for  AllianceUp  (above)  combine  to  influence  the  HoH 
to  increase  his  alliance  with  the  FNS,  and  vice  versa.  However,  the  AuthorityUp  motivations 
have  higher  volition;  the  FNS  will  care  more  about  dominating  the  HoH,  and  the  HoH  more 
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about  increasing  his  authority  over  the  LNS  in  response  to  his  prior  bad  behavior.  Taken 
together,  the  rules  that  assign  volition  to  motivations  and  moves  represent  an  agent-held  utility 
function.  Their  additive  semantics  mediate  situation  specific  influences,  while  the  specific 
volition  values  (large  or  small)  follow  authoring  conventions. 

The  intention  formation  process  utilizes  this  form  of  preference  logic  to  search  the  space  of 
situation  relevant  motivations  and  moves.  The  current  model  proceeds  top-down,  via  the 
following  steps: 

1.  It  determines  volition  values  for  all  relevant  character  motivations. 

2.  It  selects  the  top  two  motivations  for  each  character  whose  volition  score  is  above  a  pre¬ 
specified  threshold. 

3.  It  determines  the  volition  values  for  all  moves  relevant  to  those  motivations. 

4.  It  selects  the  best  move  for  each  character  whose  volition  score  is  above  a  pre-specified 
threshold. 

5.  It  passes  the  resulting  moves  (if  any)  to  the  performance  system  for  execution  (see 
Section  5.2  on  Believable  Performance). 

Intention  formation  occurs  on  a  regular  clock  -  the  characters  experience  a  decision  opportunity 
once  per  second.  In  addition  to  reasoning  about  authority,  alliance,  and  threat  state,  the  logic 
includes  some  rules  about  general  social  interaction.  For  example,  characters  are  more  prone  to 
action  if  they  have  done  nothing  for  a  time,  and  they  won’t  repeat  recent  moves  or  launch  moves 
currently  underway. 

The  final  phase  of  intention  formation  borders  on  performance  concerns,  as  it  acts  to 
parameterize  the  executable  behaviors  corresponding  to  moves.  This  reasoning  annotates  actions 
by  their  urgency  (some  intentions  form  in  response  to  events,  and  must  be  executed  quickly  to 
read  well),  or  by  the  affect  the  character  should  display  while  executing  the  move.  The 
mechanism  is  similar;  it  employs  rules  that  respond  to  social  events  and  character  properties. 

5.1.2  Social  Response 

In  addition  to  taking  autonomous  action,  characters  must  respond  to  player  input.  This  often 
leads  to  situations  where  the  characters  must  weigh  alternatives.  For  example,  when  the  player 
in  “Finding  Farah”  holds  a  photo  up  to  the  VC  as  if  to  say,  “Have  you  seen  this  person?”,  the  VC 
experiences  a  choice  between  answering,  refusing  to  answer,  and  turning  to  another  VC  for 
advice.  The  social  response  logic  generates  these  alternatives,  assigns  them  volition  scores,  and 
selects  the  best  alternative  to  pursue.  It  runs  in  response  to  a  social  signal  encoding  the  event 
that  the  player  has  asked  a  question. 

The  mechanism  is  similar  to  top-down  intention  formation,  although  we  separate  the  processes  of 
nominating  and  ranking  options: 

//  Nominate  responses,  search  over  entityMem,  bind  over  socSigMem,  assert  into  responseMem 
subgoal  nominate(nominate_decisionRequestAnswer,  entityMem,  socSigMem,  responseMem) 

//  Score  the  nominated  responses 

subgoal  score(score_decisionRequestAnswer,  responseMem); 

//  The  result  is  the  highest  ranked  response 
result  =  pickBest(responseMem); 
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This  pseudocode  is  very  close  to  the  original.  The  nomination  rules  generate  response  options 
like  basic  acceptance  and  rejection,  along  with  more  nuanced  replies  such  as  reluctantly 
complying  or  looking  to  another  character  for  guidance.  These  rules  impose  hard  constraints; 
they  identify  responses  which  are  both  relevant  and  feasible  given  the  parameters  of  the  question 
and  the  state  of  the  world.  For  example,  the  option  to  warily  reject  a  stranger’s  request  will  only 
be  nominated  if  the  decider  is  wary  and  the  asker  is  a  stranger,  as  shown  below: 

rule  nominate_rejectStrangerlfWary  { 

//  find  a  social  signal  record  capturing  the  question,  and  bind  asker  and  decider  from  it 
(SocialSignalWME  sender::asker  recipient::decider) 

//  Require  the  asker  to  be  a  stranger  to  the  decider 

(CharRelationship  cl  ==decider  c2==asker  type==Relationship. STRANGER) 

//  The  decider  must  be  wary 

(CharTrait  cl  ==decider  type==Trait.WARY) 

-> 

//  create  a  performance  request 
(PerformanceRequestWME 

siu  =  SlUs.RefuseToAnswer 
charl  =  decider 
char2  =  asker 

perfDirection=PerformanceDirection.  ANGRY) 

} 

Once  options  are  nominated,  a  second  set  of  rules  assigns  preference  scores.  These  rules 
express  soft  constraints.  They  add  volition  to  a  given  option  based  on  the  traits  of  the  asker  and 
decider,  as  well  as  the  state  of  the  world  and  history  of  past  events.  For  example,  a  character  will 
be  more  likely  to  answer  the  question  if  asked  by  someone  with  authority  over  them,  or  if 
they’ve  shared  food  with  the  asker.  On  the  other  hand,  they’ll  be  less  likely  to  answer  if  they’re 
wary  and  the  asker  is  a  stranger. 

Once  scoring  is  complete,  the  system  selects  the  highest-ranked  option  and  the  corresponding 
performance  variation  executes  for  the  character. 

While  the  current  implementation  employs  calls  to  nominate,  score,  and  select  operation  to 
invoke  the  appropriate  rulesets,  we  have  recently  added  a  language  construct  that  makes  these 
calls  implicit: 

with  (policy  policy_decisionRequestAnswer)  choose  { 

subgoal  siu_refuseToAnswer(decider,  asker,  PerformanceDirection. DEFAULT); 
subgoal  siu_agreeToAnswer(decider,  asker,  PerformanceDirection. DEFAULT); 
subgoal  siu_refuseToAnswer(decider,  asker,  PerformanceDirection. ANGRY); 

} 

In  this  configuration,  the  scoring  rules  in  the  decisionRequestAnswer  ruleset  match  onto 
annotations  associated  with  the  action  options,  which  can  be  static  or  dynamically  generated  at 
run-time.  This  form  also  replaces  the  use  of  performanceRequests  with  subgoal  calls  to  the 
executable  code  that  implements  those  performances. 

Finally,  although  the  system  currently  selects  social  responses  based  on  features  of  the 
environment  and  of  the  specific  action  options,  the  reasoning  mechanism  can  access  the 
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character’s  motivations  and  associated  volition  scores.  This  would  allow  the  character’s  choice 
(above)  between  a  positive  response  and  an  angry  refusal  to  be  influenced,  for  example,  by  the 
importance  of  building  an  alliance  with  the  person  (or  VC)  asking  the  question. 

5.1.3  Composing  Social  Games 

Taken  in  isolation,  each  social  game  captures  a  family  of  practices  surrounding  a  common 
theme.  However,  taken  together,  the  games  form  a  weakly  interacting  set.  Social  games  couple 
through  social  state;  the  execution  of  an  SIU  motivated  by  one  game  may  impact  the  state 
utilized  in  another  to  nominate  and  rank  motivations  or  moves.  Moreover,  social  games  can  be 
activated  in  in  arbitrary  combinations,  so  as  the  games  are  composed,  more  interactions  are 
exposed,  resulting  in  a  form  of  emergent  social  behavior.  This  section  describes  that  interaction 
space,  first  graphically  and  then  in  terms  of  the  resulting  behavior. 

As  mentioned  earlier,  we  utilized  three  social  games  in  Breaking  Bread,  and  provided  detail  on 
Alliance  and  Authority.  The  third  game,  Threat,  concerns  the  management  of  physical  force  and 
danger.  Threat  tracks  two  key  elements  of  state:  Might,  and  PerceivedDanger.  Might  represents 
a  character’s  level  of  power  (e.g.,  is  it  armed  or  unarmed),  while  PerceivedDanger  captures  both 
the  likelihood  and  the  magnitude  of  the  force  that  others  characters  could  apply  if  a  conflict 
arises.  The  Threat  game  contains  the  standard  three  motivations,  Threat_Up,  Threat_Down,  and 
Threat_Express,  plus  moves  for  ProtectSelf,  GetHelpWithOutburster,  HelpMeOut,  AdvanceOn, 
WardThreat  and  AcknowledgeThreat.  It  also  contains  multiple  rules  for  ranking  Threat 
motivations  and  moves. 

Figure  15  provides  a  graphical  illustration  of  the  interaction  space  enabled  by  the  Alliance, 
Authority,  and  Threat  games.  This  diagram  elides  some  of  elements  of  state,  motivations  and 
moves,  and  only  shows  relational  terms  (suppressing  arguments)  in  order  to  highlight  the 
coupling  between  games.  The  top  row  of  the  figure  lists  elements  of  social  state  input  to  the 
intention  formation  process.  The  second  row  (boxed,  with  graduated  shading)  identifies 
motivations  the  characters  can  choose.  The  next  four  rows  name  moves  that  address 
motivations,  while  the  bottom  row  shows  state  altered  by  the  execution  of  moves.  The  figure  also 
sorts  the  elements  from  left  to  right  by  social  game,  in  the  order  of  Alliance  (in  purple), 

Authority  (green),  and  Threat  (pink).  The  arrows  show  data  flow  imposed  by  nomination  and 
ranking  rules,  or  by  execution  of  the  SIUs  associated  with  moves.  More  exactly,  the  arrows 
show  the  aggregate  of  the  data  flow  across  all  rules  (meaning  a  fan-in  may  represent  multiple 
rules). 

An  important  note  about  the  figure  is  that  the  data  flow  stays  mostly  in  columns;  Threat  state 
flows  through  Threat  motivations,  into  Threat  moves  and  back  to  state.  For  example, 
PerceivedDanger  effects  the  importance  of  ThreatDown,  which  impacts  the  volition  to 
ProtectSelf.  Execution  of  ProtectSelf  may  alter  the  character’s  Might  and  PerceivedDanger, 
depending  upon  how  it  unfolds  in  the  world.  (There  is  no  guarantee;  the  SIU  for  ProtectSelf 
contains  alternate  methods,  the  situation  enables  some  paths,  and  the  player  can  impact  the 
outcomes.) 
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The  diagram  also  contains  multiple  instances  of  cross-talk.  At  the  motivational  layer, 
PerceivedDanger  decreases  the  volition  of  motivations  in  all  other  games  (as  a  means  of 
highlighting  the  relevance  of  a  threat,  no  matter  its  severity).  If  a  character  has  Alliances,  their 
presence  tends  to  decrease  the  importance  of  ThreatDown  (to  counter  perceived  danger).  A 
character  with  an  Alliance  to  an  authority  figure  looks  to  that  figure  for  support  via 
ThreatExpress  (as  the  LNS  does  to  the  player  in  response  to  the  HoH’s  outburst  in  Breaking 
Bread).  While  moves  are  almost  wholly  owned  by  games,  some  rules  employ  state  from  the 
Authority  game  called  AuthorityOver  to  nominate  and  assign  volition  to  GiveAGift. 

Similarly,  Alliance  levels  are  relevant  to  HelpMeOut.  The  cross-talk  from  moves  to  state  is 
more  profound.  Within  the  Alliance  game,  completing  social  rituals  (via  GiveAGift  or  Greet) 
impacts  Face  in  the  Authority  game.  Within  Authority,  requesting  others  to  HelpMeOut  builds 
Alliance  with  the  helper,  while  EatOthersStuff  impacts  others’  PerceivedDanger  through  the 
impression  that  the  character  is  willing  to  take  aggressive  action  (as  seen  between  the  LNS  and 
the  HoH  in  Breaking  Bread).  Similarly,  the  move  to  AdvanceOn  a  character  within  Threat 
clearly  impacts  any  prior  Alliance. 


Use  or  disclosure  of  the  data  contained  on  this  page  is  subject  to  the  restrictions  on  the  title  page  of  this  report. 


Page  29 


IMMERSE  Final  Report 

Contract  Number:  W91  INF-1 2-C-0002 


Raytheon  BBN  Technologies 


5.1.4  Layered  Demonstrations  of  Social  Games 

Because  social  games  only  interact  through  shared  state  and  soft  constraints,  no  game  is  required 
to  execute  another.  Moreover,  social  games  can  be  activated  in  arbitrary  combinations;  a  given 
combination  defines  a  space  of  possible  actions,  and  an  agent  can  initiate  any  move  in  that  space 
that  is  both  motivated  and  feasible  at  any  time.  In  order  to  demonstrate  this  point,  we  changed 
the  context  for  executing  social  games  to  a  “Sandbox”  scenario  where  the  interactions  are  more 
visible  than  in  Breaking  Bread.  The  Sandbox  contains  fewer  elements;  it  supports  interaction 
between  a  player,  two  non-player  characters  (VC01  and  VC02),  and  a  handful  of  objects  on  a 
pedestal.  We  illustrate  social  game  composition  through  three,  layered  demonstrations,  viewable 
online  (Social  Game  Videos,  2014).  The  first  generates  behavior  by  Alliance,  alone,  while  the 
next  two  add  Authority  and  Threat  in  turn. 

The  interaction  based  solely  on  the  Alliance  game  unfolds  as  follows.  We  give  VC02  a  positive 
Alliance  value  towards  the  player  and  mark  the  objects  on  the  table  as  “giftable”.  After  the 
player  greets  VC02  in  a  friendly  manner,  Gifting  Appropriate  is  created  between  VC02  and  the 
player.  This  Alliance  state  motivates  a  simple  Alliance  building  move,  GiveAGift,  for  VC02  to 
offer  the  player  one  of  the  objects  from  the  pedestal. 

The  interaction  generated  by  Alliance  plus  Authority  follows  a  different  path.  We  initialize  the 
starting  state  required  by  the  Authority  game  as  follows  VC01  is  given  ownership  over  the 
objects  on  the  pedestal  by  setting  Permission  values  for  all  the  objects  to  the  highest  tier,  5,  and 
“granted”,  the  player  and  VC02  are  assigned  Permission  values  of  1  with  the  status  “granted”, 
(which  enables  them  to  freely  “look  at”  objects  in  the  environment),  and  the  player  is  marked  as 
having  AuthorityOver  VC02  and  VC02  is  given  the  character  trait,  Arrogant.  When  the  scenario 
starts,  the  player  greets  VC01,  and  again  triggers  a  friendly  gift  offer  through  the  Alliance 
mechanics  outline  above.  Meanwhile,  AuthorityUp  motivations  coming  from  VC02’s  Arrogant 
trait  make  him  touch  items  owned  by  VC01  without  permission.  VC02’s  actions  trigger  a 
permission  update  for  that  object  to  “ungranted,  pick  up”.  The  first  time  this  violation  occurs, 
VC01  gives  VC02  a  DirtyLook,  a  move  motivated  by  an  AuthorityUp  response  to  the  object 
violation.  When  VC02  picks  up  the  bread,  this  second  ownership  violation  motivates  a  stronger 
AuthorityUp  response.  VC01  angrily  asks  VC02  to  put  down  the  bread  (via 
RequestPutThatDown).  VC02  refuses  to  comply  (a  local  accept/reject  decision),  due  to  his 
Arrogant  trait.  Finally,  because  of  previous  alliance  building  through  gifting  and  greeting,  VC01 
turns  to  the  player  and  asks  for  help,  an  Authority  move  that  is  also  sensitive  to  Alliance  state. 
The  player  asks  VC02  to  stop  his  annoying  behavior,  and  this  time,  VC02  complies  (accepts) 
because  the  player  has  AuthorityOver  relation  with  him.  This  move  by  the  player  raises  his 
Alliance  with  VC01  but  reduces  VC02’s  Face. 

Finally,  adding  the  Threat  game  makes  the  fact  that  the  player  is  holding  a  rifle  salient.  In  this 
run-through,  the  player  greets  as  before,  but  then  rejects  the  offered  gift,  offending  VC01  and 
failing  to  build  Alliance.  When  the  wrangling  over  objects  starts,  VC01  does  not  look  to  the 
player  for  assistance  because  of  the  lack  of  built  up  Alliance.  Instead,  he  simply  gets  more  and 
more  upset,  and  continues  to  apply  AuthorityUp  moves  in  response  to  VC02’s  provocations. 
VC02’s  PerceivedDanger  elevates  due  to  VCOl’s  yelling  and  he  reacts  with  fear.  He  forms  the 
motivation  to  Threat_Down,  which  triggers  the  move  to  ProtectSelf,  causing  VC02  to  raise  his 
hands  in  an  attempt  to  surrender.  When  the  player  raises  her  weapon  to  take  control  of  the 
situation,  her  Might  and  the  resulting  perceptions  of  danger  motivate  both  VCs  to  ProtectSelf  in 
response  to  Threat_Down,  now  orienting  on  the  player. 
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5.2  Believable  Performance 

In  order  to  support  the  project  goal  of  training  social  interactions,  our  virtual  characters  need  to 
perform  with  a  degree  of  naturalness  that  invites  social  engagement.  This  requirement  has 
demanded  a  level  of  attention  to  producing  believable  virtual  characters  that  goes  beyond  what  is 
common  in  research  projects,  and  that  has  shaped  both  the  mechanisms  and  the  content  we  have 
embedded  in  the  performance  component  of  the  Social  Simulation  Engine.  Our  context 
emphasize  several  aspects  of  this  task:  providing  characters  with  a  social  presence,  supporting 
unscripted  real-time  interaction,  and  maintaining  character  liveness  in  the  course  of  up-close 
encounters. 

In  a  bit  more  detail,  giving  characters  a  social  presence  requires  methods  for  enabling  social 
perception,  expressing  strategies  for  accomplishing  social  goals,  and  for  communicating  social 
signals,  including  affect.  The  need  for  unscripted  real-time  interaction  implies  that  characters 
must  handle  interruptions  while  pursuing  their  own  agendas,  but  more  broadly,  our  goal  is  to 
give  the  player  so  much  control  over  the  interaction  that  its  trajectory  has  an  emergent  flow. 

This  stands  in  contrast  to  the  scripted  branch  points  and  enforced  transitions  commonly  available 
in  games,  and  it  has  influenced  us  to  express  social  strategies  in  a  reactive  coding  style,  and  to 
represent  them  in  terms  of  recombinable,  interactive  parts  (called  Social  Interaction  Units,  or 
SIUs).  The  need  for  unscripted  real-time  interaction  also  implies  that  the  characters  experience 
unpredictable  performance  demands,  e.g.,  to  pay  attention  to  a  player’s  new  gesture  while 
performing  a  heated  exchange  with  another  VC.  This  issue  led  us  to  develop  a  performance 
management  system  charged  with  producing  smooth  interactions  by  allocating  body  resources  to 
performance  requests  as  they  arrive.  This  framing  of  the  problem  is  novel  relative  to  the  gaming 
industry.  Finally,  the  goal  of  maintaining  character  liveness  in  the  course  of  up-close  encounters 
has  influenced  us  to  develop  new  techniques  for  modeling  attention  shift  and  split  attention,  as 
well  as  to  adopt  industry  standard  practices  for  communicating  basic  character  liveness  (e.g., 
layering  in  animations  for  breathing,  idling,  blinking,  and  other  affective  stance). 

The  following  sections  discuss  our  approaches  to  these  problems  in  more  detail.  We  discuss 
social  perception,  social  interaction  strategies,  performance  guidance  (including  expression  of 
affect),  performance  management,  and  attention  management  in  turn.  We  end  with  a  brief 
discussion  of  the  underlying  agent-oriented  programming  language,  called  ABL. 

5.2.1  Social  Perception 

The  task  of  supporting  character  social  perception  requires  surmounting  an  abstraction  gap.  The 
system  needs  to  map  a  stream  of  physical  gestures  recognized  by  the  MiBA  front  end,  like 
“EXTEND  HAND”  and  “POINT-AT”  into  social  signals  like  “Request  Answer”  and  “Request 
Go  To”  that  enable  appropriate  action  in  context.  This  transformation  requires  addressing  a 
number  of  issues  common  to  sensor  processing  tasks  such  as  regularizing  noisy  input,  using 
expectations  to  resolve  ambiguity  in  perception,  and  assigning  meaning  to  observations  given  the 
character’s  current  activities  and  physical  context. 

Figure  16  illustrates  the  Social  Simulator’s  pipeline  for  social  perception.  It  begins  with 
primitive  events  recognized  by  the  MiBA  system  from  player  input.  A  pre-processing  phase 
(not  shown)  cleans  this  data  by  suppressing  repeated  and  unstable  signals,  and  augments  it  with  a 
few  skeletal  motion  features  not  native  to  MiBA.  This  results  in  a  stream  of  Embodied  Events 
capturing  gestures,  and  recognized  speech  phrases,  as  well  as  a  small  set  of  prosodic  features  and 
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facial  expressions.  Table  1  lists  the  full  set  of  primitive  features  utilized  by  the  social  simulator 
(which  does  not  employ  all  signals  generated  by  MiBA). 


Figure  16:  The  Architecture  for  Social  Perception. 
Table  1.  Primitive  Embodied  Events 


BECKON 

BOW 

EXTENDJHAND 

HAND_OVER_HEART 

HAND_TO_MOUTH 

HEAD_NOD 

HEAD  SHA 

KE 

LOOK_AT_HAND 

POINT_AT_HAND 

POINT_AT_SELF 

POINT_TO_EYE 

POINT_DOWN 

POINTJJP 

POINT_LEFT 

POINT_RIGHT 

POINT_AT_X 

REQUEST_CALMDOWN 

REQUEST  STEP 
BACK 

REQUEST 

STOP 

WAVE 

FACEJHAPPY 

FACE_FROWN 

FACE_SURPRISED 

VOCALICS_CALM 

VOCALICS 

EXCITED 

START_GESTURING 

END_GESTURING 

START_SPEECH 

END_SPEECH 

GAZE 

GAZE  PRO 
LONGED 

RUSH_FORWARD 

UTTERANCE 

YELL 

USER_SKELETON_LOST 

USER_FAR_AWAY 

The  subsequent  stages  of  perception  all  apply  context  dependent  transformations.  ABL 
pragmatics  reinterprets  a  single  Embodied  Event  based  on  the  character’s  current  activities.  For 
example,  the  rule: 

rule  EXTEND_HAND(player,  recipient)  { 

!(isOffering(recipient)) 

(isOffering(otherVC)) 

-> 

EXTEND_HAND(player,  otherVC) 

} 

re-targets  the  player’s  EXTEND_HAND  gesture  to  a  VC  who  is  offering  the  player  an  object  at 
that  moment.  In  contrast,  Compound  Sensors  deduce  new  Embodied  Events  from  sequences  of 
Embodied  Events  over  time.  For  example: 

POINT_AT_HAND  +  POINT_TO_EYE  +  holding(player,  photo)  -> 
COMPOUND_REQUEST_ANSWER 

Here,  the  point  at  hand  and  the  point  to  eye  can  be  separated  by  a  few  seconds,  while  their 
merger  into  a  single  output  signal  tells  the  characters  that  the  player  is  requesting  information 
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about  the  photo.  Table  2  lists  the  full  set  of  Embodied  Events  output  by  these  two  categories  of 
transformations. 


Table  2:  Embodied  Events  Generated  By  ABL  Pragmatics  and  Compound  Sensors 


LOOK_AT_OBJECT_IN_HAND 

POINT_AT_OBJECT_IN_HAND 

PHYSICALLY_DISENGAGED 

SOCIALLY_DISENGAGED 

COMPOUND_BECKON 

COMPOUND_USER_GONE 

COMPOUND  REQUEST 
ANSWER 

COMPOUND  REQUEST 
GO_TO 

COMPOUND  REQUEST 
STEP_BACK 

COMPOUND  POINT  AT  SELF 
_UNRECOGNIZED_SPEECH 

COMPOUND  POINT  AT 

SELF  RECOGNIZED 

SPEECH 

Several  of  these  events  include  parameters,  e.g.,  that  identify  the  recognized  speech  phrase,  or 
the  character  asked  to  step  back. 


The  next  stage  of  perceptual  processing  maps  Embodied  Events  into  Social  Signals  that  have 
direct  consequences  for  VC  action  and  social  choice.  For  example,  a  daemon  waiting  on  the 
Embodied  Event, 

COMPOUND_REQUEST_ANSWER(player,  VC,  Photo) 
generates  a  requestAnswer  Social  Signal: 

(SocialSignalWME  type=Request  value=Answer  sender=player  recipient=VC01  ...  parameters) 

It  also  launches  the  Social  Interaction  Unit  for  requestOther_answer(player,  VC01),  which 
expresses  and  manages  the  corresponding  performances.  For  example,  requestOther_answer 
includes  a  daemon  (one  of  many)  that  waits  on  the  Request- Answer  Social  Signal,  and  reacts  by 
phrasing  a  decision  to  answer  the  question,  not  answer,  reluctantly  comply,  or  look  to  another 
VC  for  guidance,  as  discussed  in  the  section  on  Social  Response  (above). 

Social  Signals  instrument  action-oriented  communications  between  the  player  and  the  virtual 
characters,  but  they  also  support  social  communications  between  VCs.  For  example,  VC01  can 
post  an  EMOTION  Social  Signal  indicating  anger  at  VC02.  That  signal  plays  into  VC02’s 
choice  of  reactions.  A  given  Social  Signal  is  typically  comprised  of  a  social  signal  type  (broad) 
and  a  value  (specific).  While  some  of  these  signals  have  function  vs  socially-oriented  names,  all 
feed  into  VC  social  choices  and  action.  Table  3  gives  the  complete  list  of  Social  Signals  types 
(excluding  their  values)  utilized  in  the  Finding  Farah  demonstration. 


Table  3:  Social  Signals  in  Finding  Farah 


EMOTION 

ENCOURAGE 

OBJECT  TRANSACTION 

OBJECT  SELF  USAGE 

OBJECT  SHOW 

REQUEST 

RITUAL 

THREATEN 

SCENARIO_KNOWLEDGE 

A  final  stage  of  social  perception  consists  of  daemons  that  monitor  social  signal  transmission  and 
induce  changes  to  social  state.  Inferences  of  this  form  perform  agent  belief  maintenance.  For 
example,  one  daemon  lowers  alliance  (player,  VC01)  in  response  to  the  Social  Signal  encoding 
VCOl’s  Request-Refusal1. 


1  To  be  precise,  the  value  of  alliance(player,  VC01)  represents  the  system’s  model  of  the  player’s  perceived  alliance 
with  VC01.  It  may  or  may  not  correspond  to  the  player’s  actual  attitude. 
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5.2.2  Expressing  Social  Interaction 

The  performance  side  of  the  architecture  in  Figure  16  enacts  social  moves  by  executing  social 
behaviors  written  in  a  special  purpose,  agent-oriented  language.  We  have  defined  an 
intermediate  vocabulary  of  these  behaviors,  called  Social  Interaction  Units  (SIUs)  that  share  two 
organizing  principles.  They  represent  basic/minimal  social  transactions  between  individuals, 
analogous  to  beats  in  Facade  (Mateas  &  Stem,  2005)  or  social  exchanges  in  Prom  Week  (McCoy 
et  al.,  2013).  SIUs  are  also  short;  they  control  character  interactions  with  each  other,  or  the 
player,  for  at  most  1-10  seconds. 

SIUs  also  share  a  common  structure  because  they  represent  interactions  between  characters.  As 
a  result,  each  SIU  consists  of  two  parallel  parts  that  describe  the  behavior  of  the  initiator  and 
responder  of  that  interaction.  For  example,  Table  4  illustrates  the  top-level  structure  of  the  SIU 
to  request  another  agent  to  show  an  object. 


Table  4:  Parallel  Structure  of  the  Request  Other  Show  Object  SIU 


VC  Initiator 

VC  Responder 

Animation:  Point  to  object 

Speak  dialog  context:  “What  do  you  have  in  your  hand  there?” 

Animation:  Look  at  requester 

Here,  the  initiator  acts  while  the  responder  pays  attention  or  otherwise  bides  time  (a  pattern  that 
repeats  across  many  SIUs).  While  this  table  illustrates  an  interaction  between  two  VCs,  the 
initiator  or  responder  could  be  the  player.  For  example,  if  the  player  is  the  initiator,  the 
corresponding  pseudocode  (Table  5)  looks  for  a  social  signal  created  by  the  player’s  action  via 
the  social  perception  pipeline. 


Table  5:  Structure  of  the  Request  Show  Object  SIU  when  the  Player  is  the  Initiator 


Player  Initiator 

VC  Responder 

Wait  for  Request  Object_Show  Social  Signal 

Absorb  (delete)  that  player  signal 

Animation:  Look  at  requester 

Every  SIU  that  involves  a  player  and  a  VC  has  an  analogous  structure.  When  the  initiator  or 
responder  is  a  VC,  the  code  causes  VC  action,  and  when  that  role  is  fulfilled  by  the  player,  the 
code  recognizes  (and  generally  consumes)  the  corresponding  Social  Signal  (if  any). 

The  Show  Object  SIU  in  Table  6  provides  a  slightly  more  elaborate  illustration  of  this  symmetric 
encoding.  Here,  a  VC  can  fulfill  either  role,  while  the  player  can  fulfill  at  most  one.  This 
example  corresponds  to  the  case  where  the  responder  complies  with  the  request  to  show  the 
object  (the  SIU  contains  another  case  that  performs  a  refusal). 


Table  6:  Symmetric  Encoding  of  the  Comply  Case  within  the  Show  Object  SIU 


VC  Initiator 

VC  Responder 

Walk  to  /  turn  to  responder  (showee) 

Initially  show  the  object 

Animation:  Object-specific  show  gesture 

Speak  dialog  context:  “Here  it  is,  take  a  look.” 

Keep  showing  the  object 

Animation:  Object-specific  show  gesture  loop 

Initial  alert  glance  at  initiator’s  (shower’s)  shown 
object 

Animation:  Alert  noticing  posture,  Walk  to 
responder  (if  needed) 

Head  tracking:  Shower’s  shown  object 

Facial  expression:  Shocked,  then  Stony 

Stop  speaking  (go  silent)  for  a  moment,  then 

Speak  dialog  context:  “Hmm.” 
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Glance  between  initiator  (shower)  and  shower’s 
shown  object 

Animation:  Alert  noticing  posture 

Head  tracking:  Alternate  between  shower  and 
shower’s  shown  object 

Player  Initiator 

Player  Responder 

Wait  for  Object_Show  Social  Signal 

Absorb  (delete)  that  player  signal 

No  Social  Signal,  nothing  to  do 

In  general,  the  internal  structure  of  an  SIU  branches  on  the  initiator  and  responder  roles,  the 
player  and  VC  role-fillers,  and  on  alternatives  where  the  responder  accepts  or  rejects  the 
initiator’s  action.  Each  of  these  units  guide  VC  behavior  for  1-10  seconds,  while  the  sequence  of 
SIUs  is  driven  by  player  action  (see  the  section  on  Social  Response)  or  by  the  top-down 
motivation  chain  initiated  by  Social  Games  (see  the  section  on  Intention  Formation).  This  results 
in  a  form  of  emergent  flow,  and  unscripted  social  interaction  (within  the  bounds  of  the 
interactions  enabled  by  SIUs). 

SIUs  also  support  two  forms  of  meta  control.  A  time-out  identifies  the  maximum  elapsed  time 
that  an  SIU  can  remain  inactive  before  it  aborts.  This  can  occur  either  because  it  is  waiting  on  a 
player  signal  that  never  arrives,  or  it  has  been  suspended  by  a  higher  priority  performance  that 
claims  its  required  resources.  A  gist  point  identifies  a  moment  in  the  execution  of  an  SIU  where 
its  purpose  has  been  sufficiently  addressed  and  communicated  to  the  user.  Any  SIU  that  fails  to 
reach  its  gist  point  will  be  retried,  from  its  start,  if  it  is  suspended  and  resumed.  In  contrast,  and 
SIU  that  has  passed  its  gist  point  will  be  considered  complete,  and  not  retried.  Time-outs  and 
gist  points  are  both  author  controlled  annotations. 

SIUs  also  have  a  hierarchical  structure  that  implements  a  form  of  functional  backchaining.  For 
example,  the  SIU  AcquireObject  (pick  up  an  object)  is  called  by  the  following  SIUs: 

•  ObjectTransaction:  for  a  VC  to  give  an  object  to  another  VC,  they  may  need  to  acquire  it  first 

•  EatObject:  for  a  VC  to  eat  an  object,  it  may  need  to  acquire  it  first  (if  not  already  held) 

•  HoardObject:  for  a  VC  to  hoard  (collect)  an  object,  it  may  need  to  acquire  it  first  (if  not 
already  held) 

•  Outburst:  to  express  emotional  outrage,  a  VC  may  want  to  acquire  an  object,  to  throw/smash 
it 

The  RequestOther  SIU  subgoals  social  purposes,  as  it  is  composed  of  two  SIUs  called  in 
sequence: 

•  InitiateRequest 

•  ComplyWithRequest  or  NotComplyWithRequest 

Similarly,  ObjectTransaction  employs  RequestOther,  and  several  additional  SIUs.  This  results  in 
the  following  subgoal  structure: 

•  ObjectTransaction 

■  RequestOther  (if  a  request  phase  is  needed  in  this  transaction) 

-  InitiateRequest 

-  ComplyWithRequest  or  NotComplyWithRequest 

■  AcquireObject 
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■  Exchange  Object 

-  OfferObject 

-  TakeOfferedObject 

■  PutDownObject  (optional)  or  EatObject  (optional) 

Table  7  lists  the  full  suite  of  SIUs  in  the  Finding  Farah  demonstration: 


Table  7:  SIUs  in  Finding  Farah 


AcquireObject 

IntroduceByName 

ReactToRushedAt 

RifleAim 

Approach 

Leave 

ReactToYell 

RifleLower 

Approached 

LookForSupport 

RejectOfferedObject 

RifleReady 

BriefAttention 

Mirror 

RequestOther 

ScaredThreatened 

BriefChat 

MirrorVocalics 

ShowObject 

Confused 

NoticeThreat 

RequestOther_AboutAnother_Attention 

StudyOwnObject 

DigestMajorMove 

ObjectTransaction 

TakeOfferedObject 

EatObject 

OfferObject 

RequestOtherAboutAnotherStopThat 

Thanks 

EnterRoom 

Outburst 

RequestOtherAnswer 

ThrowObject 

ExchangeObject 

PersonalSpace 

RequestOtherJDalmDown 

VentEmotion 

Farewell 

PlayerPicksUpObject 

RequestOtherEatObject 

Wander 

GazedAt 

PointAtHeldObject 

RequestOther_ExchangeObject 

WatchOthers 

GetAttention 

ProtectOther 

RequestOtherJ3oAway 

WhatsUp 

GoTo 

PutDownObject 

RequestOther _GoT  o 

Greet 

Question 

RequestOtherPacifyTouch 

GroupFormation 

HoardObject 

QuestionAnswerQuestion 

Question_ReactToQuestion 

RequestOther_PutObjectDown 

RequestOther_ShowObject 

RequestOther_StepBack 

RequestOther_StopThat 

5.2.3  Performance  Guidance 

There  are  many  ways  to  perform  a  given  SIU  that  corresponds  to  a  social  move.  Beyond  setting 
options  specific  to  the  SIU,  we  would  like  to  parameterize  performances  in  a  general  way  by 
applying  character  affect  to  texture  a  performance.  This  desire  relates  to  an  issue  in  authoring; 
rather  than  encode  n  versions  of  SIUs  that  capture  all  combinations  of  action  and  affect,  we 
have  developed  two  decompositional  strategies. 

The  first  is  to  define  a  vocabulary  of  affect-related  performance  directives  and  require  each  SIU 
author  to  produce  variants  that  express  at  least  a  subset  of  that  vocabulary.  For  example,  we 
employ  the  following  directives  in  “Finding  Farah”: 

•  SIU  AcquireObject 

■  PerformanceDirection.ANGRY  will  cause  the  SIU  performance  to  use  an  angry  swipe 
action  to  pick  up  an  object,  used  in  Breaking  Bread  when  the  HoH  was  going  to  pick  up  a 
bowl  and  smash  it  during  an  Outburst  SIU. 

•  SIU  AnswerQuestion 

■  When  social  games  decide  how  a  VC  will  answer  a  question  from  the  player  (e.g.,  “where 
is  the  person  in  this  photo”)  they  specify  a  performance  direction  encoding  the  tone  to 
employ  when  answering  the  question: 

-  Perfo rm ancc D  i  rcc t i  on .  W A  R  Y  is  used  if  the  AuthorityGame  decided  the  VC  felt 
affronted  by  the  questioner,,  causing  the  answerer  to  speak  in  a  halted  tone,  with  stony 
facial  expression,  with  minimal  energy 

-  PerformanceDirection.REFUCTANT  is  used  if  the  AllianceGame  decided  not  enough 
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rapport  was  built  up,  and  the  answerer  looks  nervous,  stammers,  and  doesn’t  actually 
answer  the  question 

-  PerformanceDirection. FRIENDLY  is  used  if  the  AllianceGame  decided  enough  rapport 
was  built  up,  causing  the  answerer  to  smile,  speak  in  an  upbeat  tone,  and  gesture  and 
point  excitedly 

The  wrap-on  mechanism  provides  a  novel,  and  more  general  approach  to  the  problem  of 
encoding  affect  independent  of  action.  It  operates  by  layering  expressions  of  attitude  onto 
behavior.  In  more  detail,  we  author  the  core  functionality  of  each  SIU  in  an  affect-neutral  way, 
comprised  of  very  short  functional  performances.  For  example,  Exchange  Object  has  the  initiator 
holding  out  the  object,  waiting  for  the  responder  to  reach  for  it,  which  finally  completes  the 
exchange.  Next,  we  write  behaviors  that  can  perform  these  low-level  actions  with  a  variety  of 
attitudes.  This  can  be  as  simple  as  defining  attitudinal  variations  of  individual  animations,  such 
as  holding  out  an  object  in  a  friendly,  aggressive  or  hesitant  way.  The  wrap-on  manager  acts  by 
inserting  these  animations  before,  after,  and/or  in  place  of  the  core  performance,  such  as 
impatient  sighs,  frustrated  gestures,  or  nervous  glances.  The  implementation  employs  what  are 
called  “meta-behaviors”  in  ABL;  behaviors  that  inspect  and  alter  the  activities  in  the  current 
behavioral  tree. 

We  have  implemented  a  number  of  wrap-ons  for  expressing  character  attitudes.  Characters  can 
be  hostile,  suspicious,  friendly,  sympathetic,  frustrated,  happy,  and  nervous,  and  these  attitudes 
can  each  be  applied  to  multiple  SIUs.  More  exactly,  the  object  of  a  wrap-on  is  a  fine-grained 
behavior  within  an  SIU  that  invokes  an  animation,  specified  by  its  class.  Each  {wrap-on, 
behavior}  pair  has  multiple  realizations,  expressed  by  insertions  of  a  preamble  step,  a  postlude, 
and/or  a  specific  instantiation  of  the  animation  class.  For  example,  a  hostile  instance  of  returning 
the  picture  could  involve  a  pause  (preamble),  a  rapid  extension  of  the  hand  (the  instantiation), 
and  a  shake  (postlude),  or  a  cough,  a  sneer  plus  a  slow  extension,  and  a  rapid  retraction  of  the 
hand  in  sequence.  Wrap-ons  apply  to  behavior  instances,  so  a  VC  can  greet  one  character  with 
hostility,  and  another  in  a  friendly  way. 

5.2.4  Performance  Management 

VCs,  acting  as  virtual  humans,  need  to  perform  multiple  actions  at  the  same  time.  Depending  on 
what  those  actions,  are  and  what  parts  of  the  body  they  require,  actions  can  sometimes  be 
performed  in  parallel.  However,  if  actions  conflict  over  resources,  they  must  be  performed  in 
sequence  over  time,  in  some  order  that  depends  upon  their  importance.  Actions  may  also  be 
abandoned  if  they  are  delayed  to  the  point  that  they  are  no  longer  important/relevant  anymore. 

The  Social  Simulator  contains  a  component,  called  the  performance  manager  that  mediates 
among  action  requests.  It  implements  the  following  task: 

Given: 

•  A  set  of  unfulfilled  {performance,  resource  requirement}  tuples. 

•  A  subset  of  those  performances  executing  at  the  current  time. 

•  A  decision  opportunity,  defined  as  the  arrival  of  a  new  request,  or  the  completion/termination 
of  an  existing  request. 

Choose: 

•  A  set  of  requests  to  execute,  and  a  set  of  requests  to  suspend. 
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The  algorithm  imposes  a  strict,  priority  ordered  resource  allocation  scheme.  We  discuss  the 
algorithm,  and  two  methods  for  assigning  priorities  to  performance  requests,  below. 

5.2.4. 1  Priority  based  resource  allocation 

A  performance  needs  some  subset  of  the  body’s  resources,  to  express  its  performance.  We  have 
divided  the  body  up  into  the  following  resources: 

•  LegsAndSpine  (aka  BaseFullBody) 

•  ArmLeftLayer 

•  ArmRightLayer 

•  HeadTrackingLayer 

•  SpeechLayer 

•  HeadLayer 

•  FacialExpressionLayer 

•  BodyPosture 

•  FeetPosAndBodyHeldWeight 

The  more  “efficient”  a  performance  can  be,  i.e.  the  fewer  body  resources  it  requires  to  express  its 
meaning  or  accomplish  its  task,  the  more  likely  it  is  that  other  performances  will  be  able  to  mix 
in  with  it  and  enact  simultaneously  (due  to  the  absence  of  resource  conflicts).  Our  algorithm 
distinguishes  two  types  of  resources:  necessary,  and  optional.  Behavior  authors  annotate  every 
performance  with  the  body  resources  it  truly  requires,  and  the  resources  that  would  be  nice  to 
have  (aka  optional).  In  more  detail,  each  performance  is  annotated  by  required  resources, 
optional  resources,  a  timeout  of  how  long  to  wait  for  resources  before  giving  up,  and  whether  to 
retry  if  interrupted.  For  example  a  brief  performance  to  acknowledge  that  a  player  has  spoken  to 
a  VC  (i.e.,  turning  to  look  and  saying  “hmm?”)  receives  the  following  annotations: 

performanceWME.requires(baseFullBody); 

performanceWME.requires(headTracking); 

performanceWME.requires(speech); 

performanceWME.setTimeout(3000); 

performanceWME.setBRetry(true); 

Given  data  of  this  form,  the  resource  allocation  algorithm  proceeds  via  the  following  steps: 

1.  Sort  all  new/active  performances  by  their  performance  priority,  putting  the  highest 
priority  ones  first,  and  among  priority  ties,  putting  the  oldest  performances  first.  Delete 
suspended  performances  that  have  timed  out. 

2.  Iterate  down  the  list  of  priority-sorted  new/active  performances.  If  all  of  the  required 
resources  for  a  given  performance  are  available  (i.e.,  none  of  them  have  been  claimed), 
then  claim  all  of  those  resources  for  that  performance.  (That  performance  is  now 
considered  “performable”.)  Otherwise,  claim  none.  Continue  iterating  down  the  list  of 
priority-sorted  new/active  performances;  at  least  one  performance  (the  first  on  the  list) 
will  have  received  all  of  its  required  resources. 

3.  Iterate  again  down  the  list  of  priority-sorted  new/active  performances,  considering  only 
the  performances  that  either  a)  got  all  of  their  requested  required  resources,  or  b)  had  no 
required  resources,  only  optional  ones.  If  any  of  the  optional  resources  for  a  given 
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performance  are  available,  allocate  them  to  that  performance.  This  iteration  may 
terminate  without  claiming  any  optional  resources. 

4.  If  a  performance  has  lost  its  required  resources,  and  was  marked  to  not  retry  if 
interrupted,  delete  that  performance. 

In  summary,  each  performance  receives  all  of  its  required  resources  or  none,  and  if  successful,  it 
additionally  receives  some  to  all  of  its  optional  resources.  The  (possibly  multiple)  winners  of 
this  competition  are  allowed  to  execute  animations  on  the  virtual  body,  and  all  other 
performances  are  suspended  (if  they  were  running),  or  queued  until  their  requested  resources 
become  available.  This  algorithm  runs  whenever  a  new  performance  request  arrives,  or  an 
existing  performance  ends. 

On  completion  it  is  possible  that: 

•  A  new  performance  will  not  have  received  its  required  resources,  and  will  be  queued  until 
such  time  as  they  appear. 

•  An  existing  performance  will  have  lost  its  required  resources  to  a  new,  higher  priority 
performance  with  overlapping  requirements,  and  will  therefore  be  suspended,  or  deleted  if 
marked  not  to  retry. 

•  A  new  or  existing  performance  will  start  or  restart  its  performance. 

•  Many  performances  will  receive  resources,  and  subsequently  execute  in  parallel. 

5.2A.2  Fixed  Prioritization 

The  current  suite  of  IMMERSE  demonstration  systems  all  employ  a  fixed  prioritization  model. 
In  it,  behavior  authors  assign  a  static  priority  to  each  performance  for  use  by  the  resource 
allocation  algorithm.  We  have  grouped  these  priority  annotations  into  tiers: 

•  Very  high  priority  performances 

■  scaredThreatened 

■  rifle  use 

•  High  priority  performances 

■  briefAttention  (noticing  a  new  request  from  the  player) 

■  responding  to  a  request  from  the  player 

■  requesting  something  emotionally  charged,  e.g.  request  putObjectDown 

■  ventEmotion,  throwObject 

•  Medium  priority  performances 

■  objectTransaction 

■  introduceByName 

■  greet 

■  thank 

■  eatObject 

•  Low  priority  performances 

■  briefChat 

■  groupFormation 

■  mirror 
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■  whatsUp 

•  Minimum  (background)  priority  performances 

■  CarryMgr  (right  arm) 

■  EmotiveStance  idling  (all  but  right  aim) 

While  these  assignments  reflect  a  mixture  of  general  and  scenario  specific  concerns  about 
desired  flow,  we  have  been  able  to  employ  this  fixed  priority  model  to  produce  performances 
that  read  well  in  all  of  the  interactive  experiences  we  have  demonstrated  to  date.  That  said,  the 
categorization  is  clearly  a  bit  ad  hoc,  and  it  is  unclear  if  the  use  of  fixed  priority  tiers  can  scale. 
Our  second  approach  to  assigning  performance  priorities  addresses  these  issues. 

5.2.4.3  Linear  Prioritization 

Our  second  and  more  general  approach  to  prioritizing  performance  requests  takes  a  decision 
theoretic  view.  It  defines  priority  in  terms  of  a  multi-attribute  utility  function  that  combines 
features  of  interest.  We  employ  a  linear  function  of  the  form 

v  =  W  .  F 

where  W  is  a  vector  of  weights  in  the  range  (0 , 1),  and  F  is  a  vector  of  features,  also  normalized 
to  the  (0,  1)  range. 

Before  examining  the  structure  of  F,  it  is  important  to  clarify  the  purpose  of  the  value  function 
v(W,F);  it  exists  as  input  to  the  performance  manager  for  use  in  interleaving  performance 
requests.  As  a  result,  v(W,F)  must  generate  smooth/readable  performances  and  select  against 
unnatural/jarring  performances.  The  importance  of  a  given  performance  to  the  character  is  only 
one  element  of  the  mix. 

We  have  engineered  two  classes  of  features  into  the  vector  F;  extrinsic  features  capture  the 
purposes  a  performance  serves,  while  intrinsic  features  capture  properties  of  an  individual 
performance  that  are  material  to  smooth/readable  flow.  The  extrinsic  and  intrinsic  features 
include: 

•  Extrinsic  features: 

■  Utility  of  the  action's  results  to  the  character. 

■  Dramatic  value  of  the  action  for  the  user's  experience. 

■  Importance  of  the  action  to  the  scenario. 

■  Other  application  value  of  the  action  (e.g.,  its  pedagogical  value  if  in  a  tutoring  system). 

•  Intrinsic  features: 

■  Continuity  of  the  performance  (relevance  to  previous  action,  e.g.,  if  it  is  a  part  of  a  multi- 
step  sequence). 

■  Urgency  of  the  performance  (time  window  available  for  initiating  action  if  it  is  to  read 
well  to  observer). 

■  Duration  of  the  performance  (time  required  to  execute  the  action;  a  surrogate  is  whether 
the  action  requires  locomotion). 

■  Autonomic  quality  of  the  performance  (the  degree  to  which  an  action  is  involuntary,  e.g., 
an  emotional  display). 
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■  Inertia  of  the  performance  (extent  to  which  the  action,  once  initiated,  should  not  be 
interrupted  to  read  well  -  walking  is  an  example,  as  it  feels  unnatural  for  a  character  to 
stop  and  start  a  walking  motion  multiple  times). 

We  have  implemented  this  linear  prioritization  model  and  demonstrated  its  use  in  a  stand-alone 
excerpt  of  the  Finding  Farah  scenario.  Here,  the  two  VCs  each  have  an  agenda,  while  the  player 
interjects  a  request  at  a  critical  time: 

•  Yunos  attempts  to  give  fruit  to  the  player.  This  launches  GiveGift:ObjectTransaction(Yunos, 
Player,  Fruit),  causing  Yunos  to  take  fruit  from  Basuki  and  offer  it  to  the  player.  Yunos 
executes  this  strategy  until  he  takes  offense  because  the  player’s  response  never  arrives  (a 
timeout  condition). 

•  Basuki  senses  his  fruit  is  stolen.  This  launches  his  SIU  expressing  irritation  towards  Yunos: 
RequestStepBack(Yunos,  fruit). 

•  1  second  after  Basuki  begins  RequestStepBack,  the  player  shows  the  VCs  a  photo,  implicitly 
asking  them  to  identify  the  person  involved.  This  initiates  performances  for  both  VCs  to 
respond  to  the  Request- Answer  Social  Signal. 

We  have  conducted  several  experiments  in  this  setting  to  assess  the  impact  of  feature  weights  on 
perceived  performance  quality  (smoothness/readability).  In  particular,  we  annotated  the 
underlying  performance  behaviors  with  sensible,  fixed  intrinsic  feature  values  (using  the  ABL 
description  primitive),  and  annotated  each  performance  request  with  a  reasonable,  fixed, 
aggregate  value  for  extrinsic  features.  Table  8  illustrates  the  annotations  involved  (null  maps  to 
0,  short,  medium,  and  long/high  map  to  0,  .5,  and  1  respectively).  Note  that  the  value  of 
continuity  is  determined  from  context;  it  is  set  to  high  if  the  SIU  calling  this  performance  was 
itself  called  by  another  SIU. 


Table  8:  Annotations  Defining  Intrinsic  Properties  of  Performance  Behaviors 


Urgency 

Duration 

Autonomic 

Inertia 

acquireObject approach 

null 

long 

null 

medium 

acquireObject pickUp 

null 

short 

null 

medium 

requester  RequestStepBack 

medium 

short 

medium 

high 

requesteeRequestStepBack 

medium 

short 

null 

null 

offerObject giver 

null 

long 

null 

medium 

rejectOfferedObject giver 

medium 

short 

medium 

high 

showObiect  showee  initialGlance 

medium 

long 

null 

medium 

notComplyWithRequest requesterRequestStepBack 

medium 

short 

medium 

high 

notComplyWithRequest requesteeRequestStepBack 

medium 

short 

null 

high 

showObject showee 

medium 

medium 

null 

low 

The  experiments  demonstrate  the  following  effects: 

•  No  fixed,  priority-independent  execution  policy  generates  high  quality  perceived 
performances.  This  includes  variants  on  FIFO,  and  LIFO  strategies. 

•  A  selected  weight  vector  W  can  generate  high  quality  performances,  equivalent  to  the  use  of 
hand-authored  priority  tiers.  Anecdotally,  that  weight  vector  W  was  easy  to  find.  This  is  the 
base  case. 

•  The  extrinsic  features  (taken  in  the  aggregate)  are  material  to  perceived  performance  quality. 
As  Wextnnsjc  varies  from  0  to  1  (with  all  other  weights  fixed  from  the  base  case) 
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performance  quality  alters  significantly. 

•  Each  intrinsic  feature  is  material,  on  its  own,  to  perceived  performance  quality  (via  the  same 
procedure,  as  above). 

In  more  detail,  the  FIFO  policy  creates  non-responsive  characters;  Yunos  executes  his  agenda  to 
offer  fruit  to  the  player  and  ignores  Basuki’s  irritation  until  that  sequence  completes.  Similarly, 
Yunos  fails  to  react  to  the  player’s  photo  show,  since  too  much  time  has  passed  before  he’s  free 
to  perform  the  reaction.  In  contrast,  the  FIFO  policy  creates  highly  distractible  characters; 
Basuki’s  complaint  and  Yunos ’s  corresponding  reaction  are  interrupted  mid-sentence  to  respond 
to  the  photo. 

If  the  weight  on  extrinsic  features  is  elevated  relative  to  the  base  case,  the  characters  treat  the 
player  as  overly  important.  This  causes  Basuki  and  Yunos  to  interrupt  their  argument  midstream 
to  look  at  the  photograph,  which  appears  unnatural.  Elevated  continuity  weight  favors  the 
completion  of  full  performance  sequences,  and  approximates  the  FIFO  policy.  As  a  result, 

Yunos  performs  the  full  fruit  offering  sequence,  ignoring  both  Basuki’s  complaint  and  the  photo 
show.  He  belatedly  responds  to  Basuki’s  complaint  after  the  offer  is  rejected.  Elevated  urgency 
weight  favors  reactionary  performances.  As  a  result,  both  VCs  promptly  react  to  the  player’s 
photo  show,  which  delays  the  non-urgent  fruit  theft  and  resultant  complaint  sequence.  The  loss 
in  perceived  performance  quality  is  relatively  small,  although  the  issue  is  once  again  that 
characters  have  difficulty  pursuing  longer  term  agendas.  Elevated  inertia  weight  favors 
performance  completion  without  interruption.  Basuki  performs  his  full  photo  glance  before 
reacting  to  the  fruit  theft,  which  is  clearly  unreasonable. 

Depressed  duration  weight  removes  the  priority  advantage  given  to  short  (non-locomoting) 
performances.  In  the  base  case,  after  his  initial  approach,  Yunos  chooses  to  pick  up  the  fruit 
instead  of  looking  at  the  photograph  in  part  because  the  fruit  pickup  is  short  (stationary)  while 
the  photo  investigation  is  long  (it  involves  a  walk  to  the  player).  With  duration  weight  set  low, 
Yunos  chooses  to  examine  the  photo  first  instead,  and  walks  to  the  player.  After  the 
examination,  he  resumes  the  theft  sequence,  which  requires  returning  to  Yunos.  This  results  in 
extra  locomotion,  and  a  slight  decrease  in  perceived  performance  quality. 

A  depressed  autonomic  feature  weight  causes  characters  to  be  jarringly  non-responsive.  To 
illustrate  this  case,  we  added  an  action  to  the  experimental  scenario  where  the  player  rushes  at 
the  VCs,  and  the  VC’s  startle  reaction  has  a  high  autonomic  feature  value.  In  this  situation,  the 
VCs  did  not  respond  to  the  player’s  rapid  advance  until  the  3rd  attempt. 

Our  work  on  IMMERSE  has  introduced  an  approach  to  unscripted  interaction  with  virtual 
characters  that  relies  on  satisfying  dynamically  generating  performance  requests.  This 
framework  is  novel  with  respect  to  the  gaming  industry,  and  most  virtual  character  research.  Our 
use  of  a  value  function  to  generate  aesthetically  pleasing  performance  sequences  is  similarly 
novel.  Our  preliminary  experiments  have  shown  that  a  linear  model  composed  of  extrinsic  and 
intrinsic  features  has  the  expressive  power  to  generate  nuanced  performances,  and  that  the 
specific  feature  set  is  salient,  and  material  to  the  perceived  quality  of  the  resulting  behavior. 
However,  we  have  not  demonstrated  completeness  or  independence  of  the  feature  set,  and  while 
this  approach  has  the  potential  to  scale  up,  we  have  not  tested  that  conjecture.  Future  work 
should  examine  whether  behavior  authors  find  it  easy,  or  hard  to  control  the  degrees  of  freedom 
present  within  the  linear  model. 
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An  alternate  perspective  is  that  the  extrinsic  and  intrinsic  feature  sets  represent  a  natural 
vocabulary  for  describing  performances,  and  that  the  feature  values  are  determined  by 
performance  content  and  purpose.  In  this  view,  authors  delegate  aesthetic  choices  to  the 
performance  manager  by  defining  a  tradeoff  function  via  a  vector  of  weights.  If  the  feature  set  is 
adequate,  and  the  weights  reflect  author  preferences,  the  resulting  behavior  is  by  definition  good. 
In  our  experience,  authors  need  the  ability  to  override  system  decisions  in  specialized  contexts, 
but  the  approach  of  delegating  the  vast  majority  of  such  decisions  could  easily  scale. 

5.2.5  Attention  Management 

In  order  to  create  lifelike  interactive  characters  for  the  purpose  of  teaching  effective  social 
engagement,  it  is  important  to  for  the  player  to  be  able  to  assess  the  motivations  of  the  characters 
they  are  interacting  with,  from  their  gestures,  body  language  and  facial  expressions.  How  a 
character  distributes  its  attention  between  a  multitude  of  possible  subjects,  including  other 
characters  and  the  player  is  often  overlooked  in  avatar  systems,  where  characters,  quite 
unnaturally,  shift  their  attention  entirely  from  one  character  or  object  to  another. 

In  reality,  human  beings  distribute  their  attention  (and  alter  the  distribution  of  their  attention) 
based  on  a  number  of  factors,  including  mood,  status/social  position,  and  relationship  with  the 
subjects  of  attention.  In  addition,  an  instantaneous  shift  in  attention  can  be  externally  motivated 
by  a  sudden  noise  or  the  unexpected  appearance  of  an  object  in  the  characters  field  of  view,  or 
internally  motivated,  as  when  a  character  is  completing  one  task  and  moving  on  to  another. 

These  motivations  call  for  differing  attention- shift  scenarios.  A  character  (or  person)  tends  to 
communicate  an  internally  motivated  shift  by  re-orienting  their  body  before  their  head  and  eyes. 
Externally  motivated  attention  shifts  follow  the  opposite  order;  eyes,  head  and  body  move 
toward  a  new  object  of  attention  in  that  sequence.  The  speed,  or  urgency,  with  which  these 
redirections  occur  follows  the  importance  of  the  stimuli.  The  quantity  of  resource  called  into 
play  responds  in  a  similar  way.  For  example,  a  fly  at  the  edge  of  your  vision  might  command  a 
quick  flick  of  the  eyes.  A  conversation  in  the  next  room  might  draw  your  eyes,  then  head,  and 
torso  in  gradual  succession.  In  contrast,  your  decision  to  walk  to  the  refrigerator  might  become 
visible  first  in  your  feet  and  hips,  and  shortly  thereafter  in  your  head  and  eyes. 

We  have  implemented  a  rudimentary  attention  management  capability  in  the  existing  system, 
and  have  developed  several  stand-alone  prototypes  in  Unity  to  illustrate  the  basic  behavior.  The 
most  recent  accepts  the  urgency,  intensity,  location,  and  internal/external  motivation  of  a  stimuli 
as  input,  and  then  executes  an  attention  shift  while  recruiting  an  appropriate  amount  of  body 
resources  into  the  action  as  a  function  of  time. 
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6.  Social  Pedagogy 

In  IMMERSE,  social  skills  are  fostered  through  guided  practice  within  a  socially  rich  virtual 
environment  that  presents  authentic  and  varied  social  encounters.  A  social  coaching  component 
supplements  the  experiences  available  in  the  social  simulation  to  produce  more  effective  training 
as  well  as  more  efficient  training,  an  important  consideration  for  the  eventual  integration  of 
social  competency  training  into  military  and  law  enforcement  training  regimens. 

6.1  Designing  Social  Encounters 

The  learning  experience  is  not  the  province  of  any  single  system  component,  but  arises  from  the 
interactions  of  game  play;  the  social  simulation’s  ability  to  generate  believable  and  engaging 
characters,  behaviors  and  situations;  and  the  pedagogical  component’s  ability  to  deliver  timely, 
apt  instructional  guidance.  IMMERSE  creates  social  contexts  that  go  well  beyond  any  structured 
scenario  approach  to  experience  design.  VCs  are  simultaneously  engaged  in  multiple  behaviors, 
possess  different  social  traits  and  motivations  and  are  able  to  initiate  interactions  with  the 
learner. 

Writing  robust  behaviors  characteristic  of  specific  social  settings  (e.g.,  approaching  strangers  to 
enlist  their  aid  in  locating  a  missing  person)  has  the  beneficial  side  effect  of  creating  natural 
forms  of  scaffolding  for  social  interaction.  This  derives  from  the  common  ground  inherent  in  all 
social  encounters  that  presupposes  both  parties  are  genuinely  involved  and  trying  to 
communicate  their  intent.  This  can  lead  naturally  to  supportive  behaviors  being  introduced, 
which  invite  the  learner  to  participate;  e.g.,  asking  to  see  a  picture  of  the  missing  person.  These 
behaviors  become  pedagogical  affordances  made  available  to  a  coaching  component,  and  play 
the  role  of  guiding  the  learner  to  desired  courses  of  action.  Another  characteristic  of  social 
encounters  -  prevalent  in  cross-cultural  settings  -  is  the  unavoidable  confusion  or  trouble  that 
can  arise  within  a  discourse.  Recognizing  trouble  and  collectively  seeking  remedy  is  integral  to 
social  competence. 

6.2  The  Social  Simulation  as  a  Training  Vehicle 

The  primary  pedagogical  impact  of  IMMERSE  is  inherent  in  the  authored  experiences  presented 
to  learners.  The  social  simulation  is  not  a  neutral  practice  environment  where  anything  can 
happen  at  any  time.  The  setting,  VC  motivations  and  resulting  behaviors,  and  considered 
reactions  to  learner  behaviors  combine  (in  sometimes  unpredictable  ways)  to  steer  the  experience 
toward  one  of  several  designed  pedagogical  outcomes.  IMMERSE  emphasizes  the  following 
elements  in  the  design  of  pedagogical  experiences: 

Framing.  It  is  important  to  “set  up  the  problem”  when  introducing  a  learner  to  an  experience. 
Learners  have  a  strong  desire  to  be  given  specific  goals  to  accomplish  (a  mission).  Military 
personnel  in  particular  expect  a  level  of  Doctrinal  validity,  consistent  with  known  tactics, 
techniques  and  procedures.  The  vagueness  of  the  Sandbox  scenario  was  a  stumbling  block  for 
many  learners,  although  it  could  be  argued  that  “just  getting  to  know  the  local  populace”  is  a 
worthy  goal  in  many  situations. 

Training  for  future  conflicts.  Scenarios  were  purposely  set  in  unidentifiable  countries  with 
unrecognizable  languages  and  unfamiliar  social  norms.  The  intention  was  to  force  learners  to 
adapt  and  not  fall  back  on  habits  and  expectations  formed  during  prior  deployments.  Experiences 
were  designed  to  require  social  competency  to  succeed. 
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Realism  (just  enough).  Behaviors  include  performance  actions  and  modes  that  make 
affordances  and  consequences  more  clear  and  poignant  than  they  might  be  in  a  natural  setting. 
Avatar  expressions  and  gestures  were  designed  for  readability  over  life-like-ness.  There  is  a 
tension  between  a  natural  desire  for  high-fidelity  appearances  and  performances  (expressiveness) 
and  pedagogical  salience. 

Modeling.  VCs  implicitly  demonstrate  efficacious  (and  counterproductive)  behavior.  Modeling 
behaviors  for  the  learner  draws  attention  to  affordances  and  encourages  mimicry  -  an  important 
meta-cultural  skill  in  its  own  right. 

Immersion.  Story,  performance  and  VC  engagement  with  the  learner  all  serve  to  motivate  the 
learner  to  continue  the  interaction  and  explore  alternative  actions  and  outcomes. 

Critical  junctures.  Critical  junctures  are  points  and  intervals  in  an  unfolding  social  situation 
where  the  learner’s  diagnosis  of  the  situation  and  course  of  action  (or  inaction)  has  a  large  impact 
of  the  outcome  of  the  situation.  Internally,  these  are  expressed  as  desired  social  state  and  co¬ 
occurrence  of  SIUs.  Though  not  the  same  as  dramatic  peaks,  they  are  brought  about  by  the 
human  designers  of  the  scenario  and  supported  by  the  pedagogical  elements  of  the  system.  For 
example,  in  Finding  Farah,  taking  advantage  of  the  proffered  fruit  to  establish  rapport  before 
getting  the  point  of  the  mission  has  a  large  impact  on  the  outcome;  i.e.,  whether  the  VCs  decide 
to  help  you.  Of  course  the  learner  may  not  even  have  realized  he  could  or  should  do  anything  at 
all  at  that  point:  he  may  be  unaware  of  the  “social  affordances”  available  in  the  situation.  Social 
Affordances  are  social  behaviors  that  will  have  specific  effects  at  specific  times.  They  can  be 
thought  of  as  options  or  opportunities  that  the  situation  makes  available. 

Consequences.  The  consequences  of  the  learner’s  action,  as  manifested  in  the  VC’s  behaviors, 
are  the  principal  form  of  feedback  in  IMMERSE.  The  responsiveness  of  the  social  simulation 
delivers  intrinsic  rewards  (and  penalties)  moment-by-moment.  Game-like  accoutrement  to  tally 
the  player’s  successes  and  failures  are  deliberately  not  part  of  the  IMMERSE  interface.  Rather 
successes  and  failures  are  made  obvious  to  the  learner  (e.g.,  the  head-of-household  in  Breaking 
Bread  throwing  the  fruit  bowl  on  the  floor).  Social  competency  hinges  on  learning  to  pay 
attention  to  reactions  and  adjusting  one’s  behavior  accordingly,  so  this  is  designed  into  the 
experiences. 

Outcomes.  Experiences  are  designed  around  a  set  of  outcomes.  In  practice,  the  behaviors  of  the 
VCs  -  though  not  scripted  in  any  literal  sense  -  do  tend  to  culminate  in  a  small  set  of  terminal 
interaction  states.  For  example,  in  Finding  Farah,  one  can  alienate  the  VCs  and  obtain  no 
information,  exert  force  to  obtain  limited  information,  or  gain  their  full  cooperation.  Possible 
outcomes  are  important  design  criteria;  they  become  the  main  instructional  “points”  for  the 
scenario. 

Replay.  Experiences  are  designed  encourage  repetition:  to  achieve  different  outcomes  or  to 
achieve  a  particular  outcome  more  economically  or  with  fewer  adverse  side-effects.  This 
hallmark  of  game-play  rewards  experimentation  and  alleviates  feelings  of  failure  that  a  single 
poorly  executed  all-or-none  training  event  can  produce.  Section  6.4  on  After  Action  Review 
shows  how  to  ensure  replay  is  a  productive  activity. 

6.3  The  Social  Coach 

Instructional  design  in  IMMERSE  involves  guiding  the  learner  through  situations  where  social 
skills  determine  the  outcome  and  consequences  are  visible.  The  SSIM  program  posed  this 
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question  to  a  cadre  of  social  scientists  in  this  form:  What  makes  a  warfighter  a  “good  stranger,” 
able  to  create  positive  social  outcomes  within  the  context  of  other  mission  objectives,  even  in  the 
face  of  unfamiliar  cultural  contexts  and  conflicting  social  norms?  One  distillation  of  their 
findings  emphasized  these  cross-cutting  skills:  interpreting  non-verbal  cues,  perspective-taking, 
managing  the  encounter,  building  rapport,  maintaining  self-control,  maintaining  self-awareness 
and  recognizing  social  affordances.  Social  encounters  in  IMMERSE  are  designed  to  foster  these 
skills;  e.g.,  observing  and  adopting  situationally  appropriate  rituals  like  greeting. 

IMMERSE  is  a  guided  practice  environment.  The  forms  of  guidance  derive  from  the  tenets  of 
Cognitive  Apprenticeship  (Collins  et  al.,  1989;  Collins,  1991),  which  offers  a  compelling 
argument  for  modeling  learning  on  traditional  forms  of  apprenticeship  -  coaching  and  modeling 
behaviors  to  guide  the  learner  experience  in  the  virtual  environment,  providing  feedback, 
scaffolding  and  fading,  and  a  graduated  exposure  to  increasingly  difficult,  complex  and  varied 
situations.  However,  the  dynamic  and  dramatic  qualities  of  the  social  simulation  have  forced  us 
to  extend  traditional  notions  of  guidance  and  feedback  usually  associated  with  coaching.  Since 
we  theorize  that  immersion  in  a  social  discourse  is  a  prerequisite  for  learning,  so  the  coach  must 
respect  the  dramatic  flow  of  the  interaction  by  not  undermining  learner  engagement.  To  avoid 
being  a  distraction,  the  social  coach  must  itself  respect  conversational  conventions;  e.g.,  through 
implicit  turn  taking  with  the  learner  and  other  interlocutors.  The  earlier  discussion  of  junctures 
also  affects  the  role  of  feedback,  and  because  learning  social  competency  involves  behavior 
change,  exposure  to  extreme  situations  and  failure  may  be  an  essential  ingredient  in  the  feedback 
process. 

Coaching  includes  speaking  to  the  learner  as  well  as  altering  the  visual  scene  by  zooming  the 
camera  to  focus  attention  on  a  particular  VC  behavior,  or  other  forms  of  highlighting  elements  in 
the  scene.  These  forms  of  explicit  coaching  are  augmented  by  the  implicit  coaching  inherent  in 
the  behaviors  of  the  VCs.  The  rich  social  simulation  and  expressive  performance  capabilities 
makes  possible  natural  forms  of  guidance  and  feedback.  VC  facial  expressions,  gestures  and 
other  embodied  behavior  can  vividly  reveal  short  and  longer-term  consequences  to  learner 
actions  (or  inaction).  Moreover,  VCs  can  model,  actively  solicit  or  suggest  actions  for  the 
learner. 

Despite  concerns  about  in-game  coaching  interfering  with  immersion,  there  are  situations  that 
warrant  such  interventions.  For  example,  when  a  learner  reaches  an  impasse,  the  coach  can 
suggest  a  course  of  action  to  keep  the  encounter  moving  forward.  We  have  the  ability  to  pause 
the  entire  simulation,  which  can  be  used  to  create  a  “tactical  pause”  to  reorient  and  scaffold  a 
struggling  learner.  Some  scenarios  have  more  distinct  scenes;  e.g.,  going  from  group  to  group  in 
a  village  square  to  locate  a  missing  person  in  Lost  Interpreter.  These  scenarios  create  natural 
gaps  in  the  story  that  allow  the  coach  to  jump  in  with  suggestions  or  praise. 

Another  kind  of  impasse  presents  itself  as  a  hesitancy  to  engage  with  the  system  at  all.  Despite 
the  framing  provided  at  the  start  of  a  session  -  treat  the  VCs  as  if  they  were  real  people, 
remember  that  they  can  see  and  hear  you  -  some  noticeable  fraction  of  users  fail  to  participate; 
they  just  stand  there  waiting  for  something  to  happen  as  if  they  are  watching  a  movie  play  out  in 
front  of  them.  Since  they  are  not  engaged,  there  is  nothing  to  lose  by  interjecting  a  reminder  that 
one  should  not  treat  the  experience  as  a  movie  to  be  watched,  but  an  experience  to  be  inhabited. 
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6.4  After  Action  Review 

Notwithstanding  this  ability  to  deliver  coaching  during  an  experience,  over  the  course  of  the 
project  increasing  emphasis  has  been  placed  on  a  brief  summative  after  action  review  over  in¬ 
game  coaching.  For  one  thing,  the  experiences  are  short  -  2-3  minutes  -  and  easily  disrupted  by 
taking  the  player’s  attention  away  from  the  action.  For  another,  increasing  effort  was  spent  on 
the  quality  of  the  VC’s  performance  -  its  expressiveness  and  readability  -  which  constitutes  the 
principal  form  of  feedback  and  guidance  for  the  player.  Finally,  the  structure  of  the  experiences  - 
the  points  they  are  trying  to  make  -  manifest  in  their  outcomes  (did  the  VCs  get  angry  at  you,  at 
one  another,  did  they  try  to  help  you  or  were  they  suspicious  of  you,  . . .) 

After  each  run-through  of  a  scenario,  the  player  is  presented  with  a  short  summary.  The  template 
for  this  AAR  is  simple:  first,  characterize  the  outcome  and  how  the  player’s  actions  might  have 
contributed  to  it;  offer  one  or  two  suggestions  for  things  to  try  the  next  time.  The  intent  is  to  take 
full  advantage  of  the  replay  opportunities  inherent  in  IMMERSE,  and  to  suggest  to  the  player  a 
small  number  of  alternative  behaviors  that  might  produce  more  desirable  outcomes  the  next  time 
through  (greet  others,  introduce  yourself,  accept  offers,  ask  yourself  who  was  more  friendly  and 
accommodating). 

6.5  Role  of  the  Pedagogic  Director  (PD) 

The  form  of  integration  between  the  social  simulation  engine  (and  the  social  coach)  and  the  PD 
was  driven  by  the  decision  to  keep  the  two  software  components  separate.  This  begs  the  question 
of  how  much  (and  what)  information  should  be  passed  to  the  PD  about  the  inner  workings  of  the 
SS,  and  led  to  the  framework  whereby  the  SS  proposes  and  the  PD  disposes.  All  pedagogically 
relevant  options  are  generated  in  the  SS,  either  as  VC  behaviors  or  explicit  coaching 
interventions.  The  PD  sorts  and  filters  these  options  and  passes  its  choice  back  to  the  SS  for 
execution.  A  shortcoming  of  this  approach  is  that  the  PD  lacks  full  insight  into  the  reasoning  of 
the  SS,  which  limits  its  ability  to  plan  a  trajectory  of  experience  for  the  learner.  In  reality,  by  the 
time  social  moves  were  proposed,  the  SS  had  made  major  decisions  about  the  possible  direction 
of  the  interaction,  leaving  few  meaningful  choices  for  the  PD. 

The  PD  comes  into  its  own  in  monitoring  a  learner’s  accomplishments  over  time.  During  the 
course  of  the  project,  SSIM  created  a  series  of  experiences  to  drive  the  underlying  technology, 
the  research  agenda,  and  explore  a  variety  of  types  of  experiences.  SSIM  never  aspired  to  create 
anything  close  to  what  would  be  called  a  curriculum.  Because  of  this,  it  was  not  able  to  benefit 
from  the  PD’s  capabilities  in  this  area.  Additional  future  work  on  SS-PD  integration  should 
explore  the  pedagogical  policy  options  that  guide  the  PD’s  decision-making;  e.g.,  weighing 
variability  for  its  own  sake  against  narrative  flow  and  believability,  or  the  utility  of  exposing  the 
learner  into  a  complex  situation  early  on  -  when  failure  is  likely  -  in  the  interests  of  shaking  the 
learner  loose  from  feelings  of  overconfidence. 
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7.  Additional  Knowledge  Products 

7.1  Gap  Analysis  and  Social  Science 

As  part  of  BBN’ s  IMMERSE  effort,  MacroCognition  assisted  in  transferring  results  from  the 
SSIM  TA1  effort  into  products  useful  for  social  simulation.  During  Phase  1  of  the  effort, 
MacroCognition  defined  13  KSAs  (Knowledge,  Skills  and  Abilities/ Attitudes)  which  were  key  to 
social  interactions  skills  and  “Good  Stranger”  behavior.  See  Appendix  13.4  for  additional 
details.  In  Phases  2  and  3  of  SSIM,  MacroCognition  conducted  a  Gap  Analysis,  seeking  to 
identify  the  cognitive,  behavioral  and  perceptual  gaps  between  the  Good  Stranger  (GS)  skills 
needed  and  those  possessed  by  the  military  personnel.  As  part  of  that  effort,  they  collected  data 
from  138  Non  Commissioned  Officers  (NCOs)  at  Ft.  Benning.  Findings  from  the  Gap  Analysis 
are  included  in  Appendix  13.4  and  include  theoretical  as  well  as  practical  implications  for 
military  training. 

7.2  Model  of  Social  Balance  (imposition/redress) 

SIFT  (Smart  Information  Flow  Technologies)  conducted  research  to  determine  how  to  best 
represent  non-  or  extra-verbal  “body  behaviors  (e.g.,  posture,  movement,  gesture,  facial 
expressions,  verbal  prosody,  tone  and  volume,  etc.)  within  the  framework  of  the  Etiquette 
Engine,  previously  developed  as  an  adjunct  to  DARPA’s  DARWARS  projects.  The  Etiquette 
Engine  is  a  computational  instantiation  and  elaboration  of  Brown  and  Levinson’s  [1]  theoretical 
model  of  politeness  and  its  role  in  shaping  and  managing  interpersonal  relationships.  It  was 
based  primarily  on  verbal  expressions  in  discourse  and  relied  primarily  on  word  usage  in 
utterances  to  trigger  and  compute  effects  in  the  turn  taking  typical  of  dialogue.  The  goal  of  this 
work  was  to  explore  how  the  Etiquette  Engine  might  be  expanded,  both  in  terms  of  its 
underlying  theory  and  in  terms  of  its  computational  representation,  to  incorporate  non-  and 
extraverbal  behaviors,  as  focused  on  in  the  SSIM  project.  The  project  final  report  can  be  found 
in  Appendix  13.3. 
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8.  Lessons  Learned 

8.1  Learner  Viewport  Window  Into  Another  World 

When  a  player  is  engaged  in  an  SSIM  interaction,  they’re  standing,  gesturing,  and  moving 
around,  just  as  they  would  in  an  interaction  with  a  real  person.  This  means  that  they  are  viewing 
the  screen  at  a  frequently  shifting  angle.  If  the  characters  maintain  eye  contact  with  the  “camera” 
and  do  not  track  the  player’s  movement,  then  the  simulation  can  feel  flat,  static,  and 
unresponsive,  to  both  players  and  observers. 

However,  if  the  characters  simply  turn  their  heads  or  eyes  to  follow  the  player’s  movement,  the 
resulting  experience  doesn’t  feel  correct.  Because  of  the  Mona  Lisa  effect,  a  character’s  gaze  is 
perceived  as  relative  to  the  observer  -  that  is,  if  characters  are  looking  straight  at  the  camera, 
viewers  in  any  position  will  perceive  them  as  making  eye  contact,  whereas  if  they  are  looking 
slightly  to  the  left  of  the  camera,  viewers  in  any  position  will  perceive  the  characters  as  looking 
slightly  to  their  left.  Thus,  the  experience  becomes  disorienting  for  the  player,  and  though 
observers  will  notice  a  greater  degree  of  responsiveness  to  the  player’s  actions,  if  they  pay  close 
attention  they  will  also  feel  as  though  the  characters  are  not  looking  in  the  correct  place. 

The  solution  we  used,  referred  to  as  “VR  mode”,  was  to  place  the  virtual  camera  at  a  point  in  the 
simulation  corresponding  to  the  position  of  the  player’s  head,  so  that  what  is  displayed  on  screen 
is  what  the  player  would  be  seeing  if  they  were  standing  inside  the  simulation,  looking  through  a 
window  (the  screen).  If  the  player  moves  to  the  left,  they  will  be  able  to  see  further  to  the  right, 
and  vice  versa.  A  character  on  screen  can  track  the  camera  with  their  head,  and  they  will  be  both 
visibly  moving  to  keep  the  player  in  view  as  they  move,  and  still  maintaining  eye  contact. 
Overall,  this  greatly  improves  the  subjective  sense  of  the  simulation  as  a  “real”,  three- 
dimensional  place. 

The  effects  of  VR  mode,  however,  are  specifically  tailored  to  the  player.  Observers  can  see  that 
the  system  is  responding  to  the  player’s  movement,  but  they  do  not  experience  the  same 
subjective  impressions.  Ultimately,  we  decided  that  the  player’s  experience  was  the  most 
important  factor,  and  that  it  was  not  worth  compromising  in  exchange  for  what  were  relatively 
minor  gains  for  observers. 

Figures  17a,  17b,  and  17c  each  represent  one  approach  to  the  problem  of  VC  gaze,  and  each 
column  represents  a  different  view  of  the  screen.  In  17a,  the  VC  keeps  their  gaze  fixed  on  the 
camera,  and  so  does  not  move  while  the  player  walks  around,  but  appears  to  be  maintaining  eye 
contact.  In  17b,  the  VC  tries  to  watch  the  player,  but  it  will  always  look  at  though  they  are 
looking  in  the  wrong  direction.  Finally,  17c  represents  “VR  mode”,  the  solution  we  used,  in 
which  the  player’s  view  of  the  simulation  moves  when  the  player  does,  as  though  they  were 
looking  through  a  window.  Note  that,  in  the  first  two  figures,  even  when  looking  at  the  screen 
from  the  side,  the  VC  is  seen  “head  on”  and  the  background  does  not  change,  whereas  in  the 
final  row,  looking  at  the  VC  from  the  side  provides  an  appropriately  angled  view. 
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Figure  17a.  Character  watches  camera 


Figure  17b.  Character  watches  player 


Figure  17c.  “VR  Mode” 


8.2  Encoding  Social  Interaction  Requires  a  Human  Model 

At  the  mechanical  level,  this  meant  that  we  acted  out  (role-played)  multiple  variants  of  our 
demonstration  scenarios  on  camera,  so  that  we  could  dissect  our  own  performances  for  social 
content  (as  well  as  animation  models).  The  rationale  is  that  social  behavior  is  so  ingrained 
(analogous  to  physical  skills)  that  our  purposes,  strategies,  social  communications  and  physical 
signals  are  not  easily  available  for  introspection.  We  found  that  we  designed  SIUs  and  social 
games  by  looking  at  what  we  did,  vs.  what  we  planned. 

8.3  Expressive  Al  System  Development  is  a  Deeply  Incremental 
Process 

We  found  that  we  needed  to  build  behavior,  see  it  on  the  screen,  and  refine  it  to  a  level  of 
expressive  quality  that  let  us  perceive  the  next  layer  of  issues  we  needed  to  address  without  being 
distracted  by  performance  artifacts.  This  process  repeatedly  delayed  work  on  general  mechanism 
in  favor  of  character  performance.  Some  version  of  this  tension  is  always  present  in  large  system 
development  activities,  but  it  has  a  novel  flavor  here.  Research  on  expressive  interaction  requires 
an  iterative,  incremental  methodology  to  be  feasible  at  all. 

8.4  Long  Duration  Pre-planned  Behavior  Sequences  Inhibit  Interaction 

The  goal  of  supporting  non-scripted  social  interactions  between  the  player  and  virtual  characters 
exerted  a  strong  influence  on  our  technical  work.  This  effect  became  apparent  when  we 
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attempted  to  employ  such  durative  strategies  in  our  early  demonstrations.  In  addition,  we  found 
that  characters  become  unresponsive  if  the  player  did  not  respond  with  any  of  the  inputs 
anticipated  by  the  character’s  agenda.  This  is  a  separate,  but  related  issue.  We  considered  the  use 
of  strongly  reactive  coding  techniques  that  would  allow  SIUs  to  be  arbitrarily  suspended  and 
resumed  under  changed  environmental  conditions,  but  concluded  that  the  desire  for  unscripted 
social  interaction  selected  for  a  more  compositional  model.  We  learned  that  our  approach  of 
assembling  interactions  from  primitive  social  elements  (motivations,  moves,  and  SIUs)  enabled 
emergent  social  behavior. 

8.5  Whole-body  Interaction  Entails  a  Profoundly  Difficult  Sensor 
Processing  Task 

Despite  a  good  deal  of  attention  to  signal  filtering,  we  had  to  place  strong  expectations  on  player 
input  to  address  issues  with  false  positives  and  negatives  in  gesture  recognition.  This  narrowed 
the  breadth  of  allowed  player  input,  frequently  in  uncomfortable  ways.  Given  more  time  and 
resources,  and  especially  in  hindsight,  we  would  have  engaged  this  issue  directly  by  employing 
stronger  filtering  methods  as  well  as  active  uncertainty  resolution  techniques.  For  example,  we 
arguably  should  have  employed  expectations  to  condition  perception  via  hidden  Markov  models, 
and  could  have  resolved  social  ambiguities  through  non-verbal  dialogue  interactions  between  the 
player  and  characters. 
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9.  Unanswered  Questions  about  Social  Interaction  with 
Virtual  Characters 

9.1  Player  Locomotion 

The  problem  of  player  locomotion  is  persistent,  and  imposes  serious  limitations  on  the  design  of 
virtual  interactions.  Our  system  has  used  several  different  control  schemes,  each  with  its  own 
benefits  and  drawbacks. 

Currently,  players  can  move  about  physically  in  order  to  control  their  location  in  the  simulation. 
This  mode  of  movement  is  very  natural  and  unobtrusive,  but  it  means  that  the  player  can  only 
meaningfully  move  within  the  space  both  visible  to  the  sensors  and  in  view  of  the  screen. 
Therefore,  scenarios  using  this  method  must  account  for  the  fact  that  the  player  will  have  to 
remain  in  roughly  the  same  overall  position,  and  will  view  the  scene  from  a  fairly  restricted  set 
of  angles  -  the  player  cannot  look  behind  themselves,  or  interpose  themselves  between  two 
characters,  for  example. 

Earlier  on,  in  the  Lost  Interpreter  scenario,  players  could  move  about  locally  just  as  they  can 
now,  but  could  also  move  their  character  using  a  hand-held  controller  with  an  analogue  stick. 

This  provided  a  much  greater  range  of  motion,  as  a  player  could  turn  around,  walk  up  to  specific 
groups  of  people,  and  otherwise  navigate  the  virtual  space  using  the  controller,  while  still  being 
able  to  make  finer  adjustments  with  natural,  physical  movements.  The  chief  drawback  to  this 
method,  apart  from  the  higher  learning  barrier,  is  that  it  is  very  unnatural.  The  dissonance 
involved  in  controlling  your  character  in  a  “game-like”  way  with  a  controller,  while  also 
attempting  to  engage  in  social  interactions  in  which  you  must  pretend  you  are  not  holding  that 
same  controller,  is  very  damaging  to  the  sense  of  immersion.  Using  two  different  methods  to 
move  about  can  feel  awkward  and  arbitrary,  and  is  overall  distracting. 

Another  solution  considered  but  not  used  in  this  project,  would  be  to  use  scripted  transitions 
between  parts  of  a  scene.  For  example,  when  you  finish  talking  to  someone,  you  might 
automatically  walk  over  to  another  group  of  people.  This  falls  into  the  category  of  methods  that 
might  serve  in  practical  terms  to  remove  the  controller  while  still  allowing  motion,  but  cannot  be 
considered  a  full  solution.  The  player  still  has  only  a  limited  range  of  motion  under  their  control, 
and  there  is  still  an  unusual,  disorienting  disconnect  between  the  two  locomotion  methods. 

VR  headsets  solve  the  problem  of  a  stationary  screen,  and  allow  the  player  to  turn  around  in  a 
natural  fashion,  but  they  make  the  problem  of  locomotion  even  more  pronounced.  It  is  more 
disorienting  if  the  character’s  movement  does  not  correspond  to  movement  of  the  player  (as 
when  using  a  controller  or  scripted  transitions),  and  a  freely-wandering  player  might  have  a 
slightly  larger  range  of  motion  (because  they  no  longer  have  to  keep  the  screen  in  view)  but  they 
must  contend  with  the  more  pressing  problem  of,  essentially,  walking  around  blindfolded. 

There  have  been  several  attempts  to  create  easy-to-use  omnidirectional  treadmills  that  would 
allow  a  player  to  walk  in  any  direction  in  a  natural  way  and  so  navigate  in  a  simulation 
(perceived  via  a  VR  headset),  without  physically  moving  around.  For  example,  the  player  might 
stand  inside  a  large,  hollow  sphere,  allowed  to  roll  freely  in  place.  Unfortunately,  these  tend  to  be 
expensive,  bulky,  and  difficult  to  use. 
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9.2  Interacting  with  Real  and  Virtual  Objects 

Interacting  directly  with  objects  in  a  virtual  space  presents  unique  challenges.  The  mismatch 
between  what  the  player  is  seeing,  what  they  are  able  to  touch  and  feel,  and  what  they  are 
intended  to  believe  is  going  on  in  the  scenario,  can  be  difficult  to  reconcile  satisfactorily.  If  the 
player  must  interact  with  virtual  objects  without  a  physical  correlate,  they  can  be  difficult  to 
manipulate.  Can  you  put  your  ID  in  your  pocket?  Move  it  from  one  hand  to  the  other?  Even  the 
most  basic  information  -  are  you  holding  something,  and  if  so,  what  -  can  be  difficult  to  convey 
and  easy  to  forget. 

Using  a  physical  object  to  match  a  virtual  one,  though,  is  an  even  trickier  territory.  Such  an 
object  could  interfere  with  sensors  intended  to  track  the  player,  and  would  require  additional 
sensing  to  determine  whether  the  player  has  swapped  hands,  put  it  away,  or  is  holding  it  at  an 
awkward  angle.  Those  difficulties,  though,  are  relatively  tractable,  and  could  be  worked  out  for 
interactions  involving  very  specific  items  that  the  player  will  keep  on  them  at  all  times.  But  what 
if  you’re  intended  to  give  the  object  to  someone,  take  it  from  them,  or  put  it  on  a  (virtual)  table? 
What  if  you  might  want  to  take  (and  eat)  multiple  pieces  of  fruit?  This  technique  can  quickly  get 
out  of  hand. 

The  technique  we  used  to  address  this  issue  is  a  visual  “inventory”  displayed  on  the  screen.  If  a 
player  picks  up  an  object,  it  is  displayed  on  screen  in  the  appropriate  hand  (as  a  somewhat 
abstracted  image).  If  the  player  extends  a  hand  with  an  object  forward,  the  inventory  image  gets 
larger.  Though  these  objects  can’t  be  put  away  or  swapped  between  hands,  there’s  a  constant 
reminder  of  what  the  players  are  holding.  Additionally,  during  development  of  the  Breaking 
Bread  scenario,  we  tested  the  use  of  a  rifle-shaped  controller.  This  scenario  managed  to  avoid 
many  of  the  above  problems  with  physical  representations  of  virtual  objects,  because  it  couldn’t 
leave  the  player’s  possession,  and  there  was  a  very  limited  range  of  sensible  actions  that  could  be 
performed  with  it  (the  scenario  ended  if  the  rifle  was  aimed  at  a  person).  All  of  these  techniques 
had  significant  tradeoffs  to  consider,  and  it  is  yet  to  be  determined  what  solution  is  best  overall. 

9.3  Animation  Combinations  and  Stage  Management 

The  animation  requirements  of  a  system  like  this  are  complex.  Whereas  normally  a  game  or 
simulation  would  be  able  to  use  finite  state  machines  to  control  the  transitions  between  specific 
animations,  and  use  perhaps  one  or  two  additional  layers  of  animations  or  inverse  kinematic 
aiming,  in  this  project  we  use  more  than  ten  distinct  conceptual  layers  and  we  permit  any 
animation  to  transition  directly  into  any  other.  A  character  might  be  walking  towards  a  table  to 
pick  something  up  when  they  notice  a  friend  of  theirs  ,  they  continue  to  walk,  look  at  the  other 
character,  smile,  nod,  wave,  and  pick  up  the  object.  More  than  that,  this  sort  of  mixing  of 
animations  must  occur  on  the  fly,  in  reaction  to  unpredictable  actions  from  the  player,  on  varying 
time  scales  down  to  the  fraction  of  a  second  (for  instance,  flinching  if  the  player  yells). 

Characters  must  also  have  some  form  of  awareness  of  their  positioning  relative  to  other 
characters,  and  establish  proper  positioning  without  distracting  from  social  interactions.  For 
instance,  they  should  make  sure  not  to  block  anyone’s  line  of  sight,  move  around  others  without 
bumping  into  them,  and  stay  at  a  socially  appropriate  distance  from  those  they’re  interacting  with 
(whatever  that  happens  to  be  at  the  time).  These  concerns  can  seem  small,  but  players  will  feel 
instinctively  that  the  interaction  is  ‘wrong’  if  characters  don’t  abide  by  them. 
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Another  layer  of  complication  is  added  by  the  fact  that  characters  must  fulfill  these  obligations 
with  a  limited  array  of  resources,  both  physical  (two  hands,  two  eyes  that  must  generally  focus 
on  the  same  thing,  etc.)  and  mental  (attention,  cognition,  perception,  etc.).  Tasks  of  varying 
importance  and  urgency  can  arise  at  a  moment’s  notice,  and  the  correct  reaction  can  depend 
heavily  on  context.  You  might  put  down  your  cup  of  coffee  to  shake  hands  with  the  President, 
but  if  you  see  a  priceless  vase  falling,  you  don’t  have  time  to  look  for  a  coaster. 

In  future  projects,  more  advanced  animation  systems  could  help  make  these  interactions  seem 
more  natural,  or,  alternately,  more  stylized  and  abstract  characters  could  be  easier  to  animate. 
Procedural  animation  is  likely  to  play  a  part,  given  the  broadness  and  context- sensitivity  of  the 
challenge.  We’ve  made  a  concerted  effort,  via  systems  of  resources,  adjustable  performance 
priority,  and  layers  of  animations,  to  allow  our  characters  to  fulfill  all  of  the  performance 
requirements  imposed  upon  them,  but  there  is  still  a  great  deal  of  work  required  in  this  area. 
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10.  Recommendations  on  the  Way  Ahead 

The  following  section  describes  several  key  areas  of  research  and  development  necessary  to 
expand  the  state-of-the-art  in  social  simulation. 

10.1  Authoring  for  Social  Simulation 

Our  work  on  the  DARPA  SSIM  program  has  taught  us  that  writing  scenarios  for  social 
interaction  simulators  is  hard  work  and  currently  requires  skilled  programmers  who  understand 
social  interaction,  automated  pedagogy,  and  DoD  mission  requirements.  Yet,  the  runaway 
success  of  Ambush!  taught  us  that  game-based  training  will  only  receive  wide  adoption  if 
training  (and  learning)  communities  can  easily  create  their  own  content.  Therefore,  new,  AI- 
supported  authoring  tools  are  needed  so  that  non-programmer,  DoD  stakeholders  can 
independently  develop  the  complex  social  scenarios  needed  for  Human  Dimension  training  and 
mission  rehearsal. 

Such  authoring  tools  will  need  to  meet  several  broad  requirements.  They  must:  1)  be  trainable, 
usable  and  satisfying  for  learners,  trainers  and  SMEs  to  use,  rather  than  requiring  software 
developers;  2)  support  a  simulation-asset  writing  and  sharing  ecosystem;  3)  implicitly  support  a 
pedagogical  orientation,  without  forcing  simplification  of  the  experiences,  challenges  and 
outcomes  that  are  produced;  4)  support  a  story-to-game  pipeline  for  rapid  translation  of 
operational  incidents  into  training  experiences;  and  5)  support  testing  and  revision. 

The  IMMERSE  project  developed  a  model  of  social  simulation  that  supports  spontaneous 
unscripted  interactions  with  groups  of  virtual  characters.  A  primary  element  of  this  approach  is 
the  concept  of  Social  Games,  described  above.  While  IMMERSE  has  focused  on  creating  this 
novel  capability,  feasibility  for  widespread  use  is  a  next,  obvious  concern;  in  place  of  relying  on 
programmers  to  write  rules  and  procedures  for  generating  character  behavior,  authors  should  be 
able  to  produce  social  pedagogy  with  synthetic  characters  via  GUI  based  authoring  in  vastly  less 
time. 

We  think  that  such  a  capability  can  be  created  and  deployed  over  the  course  of  three  to  five  years 
with  service  personnel  involvement  beginning  during  the  first  year  and  accelerating  to  full 
ownership  of  the  capability  at  the  endeavor’s  end. 

10.1.1  Prototype  Experience  Envelope  Visualization 

Toward  the  end  of  the  SSIM  program,  we  considered  how  scenario  authoring  might  work  in  the 
future,  and  wondered  whether  we  might  visualize,  and  some  day  perhaps  adjust  (via  the 
visualization),  the  full  state-space  of  everything  that  could  ever  happen  in  a  given  scenario. 
Figure  18  shows  a  rapidly  developed  prototype  (really  more  of  a  conversation  piece)  illustrating 
what  this  might  look  like,  based  on  the  Finding  Farah  scenario.  Each  red,  gray,  and  green  box 
represents  a  player  move  in  the  scenario,  while  the  outcome  of  the  move  is  represented  by  its 
color  (green  for  success,  red  for  VC  rejection,  gray  for  no  significant  state  change).  Each 
column  of  boxes  represents  enough  time  passing  in  the  scenario  for  the  player  to  perform  an 
action,  while  each  row  of  boxes  represents  one  possible  path  through  the  scenario  (a  unique 
sequence  of  actions  by  the  player).  The  “String  Length”  slider  allows  for  the  visualization  of 
more  or  fewer  player  moves,  while  the  “FruitAlliance”  slider  provides  an  example  of  changing  a 
parameter  in  the  scenario  to  see  instantly  what  effect  the  change  has  on  the  entire  state  space. 

The  blue  text  and  the  two  graphs  on  the  right  show  information  about  the  row  of  boxes  currently 
under  the  cursor:  the  text  shows  what  player  actions  the  boxes  represent,  and  the  charts  show 
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how  certain  social  state  elements  change  over  time  in  that  scenario  given  that  unique  sequence  of 
player  actions.  Finally,  the  Player  Type  menu  allows  for  the  selection  of  specific 
“quintessential”  sequences  of  player  actions  (e.g.  “Impatient  Authority- Wielding”,  “Patient 
Friendly”)  or  “U*”  to  allow  for  exploration  of  the  entire  state  space,  as  we  see  in  Figure  18. 


Figure  18:  Scenario  State  Space  Visualization  Prototype 

Turning  this  conversation  piece  into  something  useful  is  left  as  an  exercise  for  future 
development.  Obvious  issues  include  dealing  with  simplifying  assumptions  here  about  time 
quantization  and  the  number  of  relevant  player  actions,  e.g.,  as  well  as  addressing  the  fact  that 
social  state  is  naturally  far  more  complex  in  real  scenarios.  Furthermore,  the  authoring  of  the  AI 
rules  that  govern  behavior  in  the  existing  system  is  entirely  bottom-up:  specific  rules  are 
authored  by  human  developers  that  govern  agent  behavior,  and  those  rules  are  refined  until  the 
behavior  that  emerges  from  the  agents  is  what  we  hope  to  see.  The  possibility  of  using  a  state 
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space  visualizer  to  adjust  agent  behavior  suggests  the  presupposition  of  an  entirely  inverted 
approach  to  behavior  authoring  -  one  probably  based  either  partially  or  entirely  on  machine 
learning. 

10.2  Content  Based  Architectures  for  Social  Simulation 

With  the  rise  of  industry-leading  game-engines,  like  the  Unreal  Engine  and  Unity,  interactive  3D 
content-authoring  has  moved  from  the,  almost  exclusive,  domain  of  computer  programmers  to 
one  in  which  technical  artists  and  game  designers  with  little  or  no  programming  experience  can 
create  highly  sophisticated  games  and  interactive  experiences.  This  has  been  possible  by  the  shift 
toward  content-based  architectures,  in  which  assets,  and  their  behaviors  and  relationships,  are 
created  using  visual  authoring  tools  which  are  used  to  determine  the  code  that  will  be  executed  at 
run-time. 

At  present,  the  Social  Simulator  is  implemented  as  a  set  of  API’s  and  language  features  in  ABL 
that  support  the  creation  of  social  rules,  interactive  behaviors,  and  physical  performances  by 
animated  avatars.  While  this  has  facilitated  the  rapid  prototyping  of  simple  scenarios  by  software 
engineers  proficient  in  the  language,  the  lack  of  a  formalized  content  model,  independent  of  the 
language,  greatly  limits  the  ability  to  create  more  complex  scenarios,  or  to  create  tools  for 
pedagogic  experts  and  other  users  who  may  not  be  skilled  computer  programmers. 

The  obvious  next  step  is  to  apply  the  content-based  architecture  model  that  is  so  effective  in  the 
commercial,  high-end  video  game  production  to  the  task  of  authoring  social  games  and 
behaviors.  This  transformation  would  adapt  the  technology  to  preserve  many  of  the  control 
concepts  found  in  the  current  ABL  implementation.  Our  initial  work  in  this  area  suggests  that 
the  marriage  is  quite  feasible,  and  would  greatly  facilitate  the  creation  of  unscripted  social 
interaction  scenarios,  which  are  quite  novel  relative  to  game  industry  applications. 

10.3  Recombinable  Content 

While  the  content  based  model  facilitates  authoring  by  scaling  up  the  instantiation  of  behavior 
patterns,  the  patterns  themselves  must  be  defined  in  a  reusable  and  recombinable  form.  As 
discussed  earlier,  social  behaviors  have  novel  content  and  structure,  and  need  some 
generalization  to  mate  with  the  content-based  model.  Our  work  on  IMMERSE  has  solved  a 
major  portion  of  this  issue;  social  games,  motivations,  moves,  and  SIUs  are  already  expressed  in 
terms  of  simple  elements  that  can  be  utilized  in  arbitrary  orders  and  combinations.  However, 
they  have  not  been  organized  into  assets  whose  internal  structure  can  be  instantiated  and 
modified  in  stereotypical  ways. 

We  can  accomplish  this  transformation  by  refactoring  the  implementation  of  games,  motivations, 
moves,  SIUs,  and  the  component  behaviors  into  stereotypical  patterns  with  fixed  and  variable 
(author-selected)  elements.  This  involves  inventing,  and  clarifying  idioms  as  well  as  factoring 
content,  for  example,  to  separate  the  function  and  allowed  content  of  nomination  and  preference 
rules,  and  to  define  models  (idioms)  for  utilizing  magic  numbers  to  assign  volition. 

This  task  is  quite  feasible.  We  have  examined  the  problem  of  porting  a  social  game  across 
scenarios  in  detail,  including  several  subtasks: 

•  defining  a  new  move,  with  new  nomination  and  scoring  logic 

•  defining  a  new  SIU  that  must  be  mated  to  the  intention  formation  chain, 

•  specializing  an  existing  social  move  to  operate  in  a  new  context  (with  modification  of  the 
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intention  formation  logic),  and 

•  specializing  an  existing  SIU  to  take  appropriate  action  in  a  new  context. 

In  all  cases,  the  idioms  utilized  in  our  existing  work  mapped  into  the  content  based  authoring 
format  in  straightforward  ways. 

10.4  New  Kinds  of  Synthetic  Experiences 

One  clear  direction  for  future  work  is  to  build  applications  of  our  social  interaction  technology. 
In  particular,  while  this  contract  has  created  proof  of  concept  experiences  for  practicing  social 
skills  with  virtual  characters,  UCSC  and  BBN  have  recently  developed  games  for  IARPA  that 
cause  perspective  shift  by  provoking  players  to  internalize  another  culture’s  mindset.  We  can 
combine  these  threads  to  produce  a  new  genre  of  highly  effective  cross-cultural  training  games 
that  will  give  warfighters  deep  insight  into  human  dimension  concerns. 

In  more  detail,  we  learned  several  lessons  from  SSIM  and  the  IARPA  EMIC  efforts  that  suggest 
a  practical  way  forward.  Our  work  on  SSIM  taught  us  that  social  competence  can  only  be 
mastered  through  practice  in  the  context  of  dynamic,  joint  endeavors.  That  practice  can  occur  in 
simulation  or  between  people.  Second,  our  work  on  EMIC  taught  us  that  cross-cultural  training 
can  go  far  beyond  instruction  in  social  conventions  (e.g.,  bowing,  when  to  look  or  not  look 
someone  in  the  eye);  it  can  teach  deep  perspective  shift  through  visceral  first  person  experiences. 
Third,  the  EMIC  project  identified  game  design  principles  that  generate  perspective  shift.  The 
key  insight  is  to  impose  rules  on  the  interaction  that  make  culturally  foreign  perceptions, 
motivations,  and  action  options  visible  to  the  player,  and  that  similarly  discourage  non-native 
(e.g.,  American)  perceptions,  motivations,  and  responses.  These  principles  are  applicable  to 
simulation-based  games,  as  well  as  live  role-play  interactions. 

We  believe  that  we  can  apply  these  lessons  to  build  a  genre  of  training  games  that  will  give 
warfighters  nuanced  insight  into  the  mindsets  of  organizations  like  ISIS,  Hezbollah,  Boko 
Haram,  and  others,  who  will  populate  the  human  terrain  of  likely  future  battle  environments. 

For  example,  a  game  can  communicate  the  centrality  of  Sharia  law  to  the  non-Islamic  mind  by 
causing  players  to  enforce  such  laws  within  the  game,  while  receiving  approval  from  peers. 
Techniques  of  this  form  are  already  in  use  in  “bleed  LARPS”;  live  action  role-play  games 
designed  to  affect  a  player’s  thoughts  and  feelings  in  their  daily  lives. 

This  synthesis  of  ideas  suggests  two  complementary  ways  forward.  The  low-tech  path  employs 
live  role-playing  games  as  opposed  to  simulators  to  deliver  social  training.  This  approach  can  be 
delivered  at  scale  because  the  game’s  content  and  rules  communicate  the  pedagogical  material 
obtained  from  social  scientists  and  other  subject  matter  experts.  As  a  result,  warfighters  can 
learn  from  playing  the  game,  absent  an  instructor.  The  high-tech  approach  employs  social 
simulation  as  in  the  SSIM  project,  using  the  EMIC  model  to  design  perspective  shift  training. 
The  resulting  experiences  can  be  presented  in  the  form  of  a  web  or  tablet  based  game,  and 
delivered  at  scale  through  replication. 


Use  or  disclosure  of  the  data  contained  on  this  page  is  subject  to  the  restrictions  on  the  title  page  of  this  report. 


Page  58 


IMMERSE  Final  Report 

Contract  Number:  W91  INF-1 2-C-0002 


Raytheon  BBN  Technologies 


11.  Conclusion 

We  believe  that  the  BBN  team  and  its  collaborators  have  substantially  advanced  the  state  of  the 
art  in  embodied  interactions  with  virtual,  artificial  intelligence  controlled  characters  while 
moving  toward  enabling  real  social  interaction  in  a  synthetic  setting.  This  work  could  be 
exploited  to  produce  evaluatable  training  capability  for  teaching  social  competence.  Our  team 
members  would  be  more  than  happy  to  continue  in  this  direction. 

As  we  have  carried  out  our  work  for  DARPA  under  SSIM,  we  have  been  aware  of  a  parallel 
movement  inside  if  the  Army  (though  other  services  are  also  involved)  towards  defining  a 
“Seventh  Warfighting  Requirement”  which  may  be  called  “Engagement”.  Basically  this  is  the 
capacity  to  execute  missions  that  involve  non-kinetic  interactions  with  civilians  and  members  of 
other  fighting  forces  in  contexts  where  influence  and  trust  are  critical  to  mission  success  and 
where  there  may  be  ubiquitous,  embedded  enemy  elements.  These  are  operations  in  the  Human 
Domain.  As  this  (much  needed)  doctrinal  change  evolves  over  time,  we  expect  that  the  work 
done  under  the  SSIM  program  can  be  used  to  meet  the  profound  training  requirements  that  will 
be  derived  from  requirement. 

In  any  case,  we  hope  that  the  SSIM  simulator  that  we  have  constructed  and  the  insights  gained 
from  its  production  will  enable  new  solutions  to  being  ready  for  whatever  tomorrow  requires  of 
our  warfighters,  police  officers,  emergency  responders  and  anyone  else  whose  work  depends  on 
interacting  across  cultural  boundaries  in  high  stakes  social  situations. 
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13.  Appendices 

13.1  List  of  Symbols,  Abbreviations,  and  Acronyms 


AAR 

After-Action  Report 

ABL 

A  Behavior  Language 

Al 

Artificial  Intelligence 

API 

Application  Program  Interface 

AR 

Augmented  Reality 

DARPA 

Defense  Advanced  Research  Projects  Agency 

DoD 

Department  of  Defense 

GS 

Good  Stranger 

GUI 

Graphical  User  Interface 

HoH 

Head  of  Household 

IARPA 

Intelligence  Advanced  Research  Projects  Activity 

IK 

Inverse  Kinematics 

IMMERSE 

Interactive  Mentoring  for  Multimodal  Experiences  in  Realistic  Social  Encounters 

ISIS 

Islamic  State  of  Iraq  and  Syria 

KSA 

Knowledge,  Skills  &  Abilities/Attributes 

LARP 

Live-Action  Role  Play 

LN/LNS 

Local  National  /  Local  National  Soldier 

MiBA 

Multimodal  Integrated  Behavior  Analysis 

NCO 

Non-Commissioned  Officer 

NPC 

Non-Player  Character 

PD 

Pedagogical  Director 

SIFT 

Smart  Information  Flow  Technologies 

SIU 

Social  Interaction  Unit 

SME 

Subject-Matter  Expert 

SP 

Social  Pragmatics 

SS 

Social  Simulator 

SSIM 

Strategic  Social  Interaction  Modules 

TA1, 2,  3 

Technical  Area  1 , 2,  3 

UCSC 

University  of  California,  Santa  Cruz 

USA 

United  States  of  America 

VC 

Virtual  Character 

VR 

Virtual  Reality 

WME 

Working  Memory  Element 
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1.  Document  History 
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Sonja  adds  some  text  from  literature  review 
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section. 

.5 

11/7/14 

Cleaned  up  section  7.  Started  on  section  8. 

.6 

1 1/20/14 

New  version,  since  v05  seemed  corrupt.  Sonja  finished 
integration  of  “nonverbal  behavior”  at  the  end  of  section  7 
(which  should  be  moved  to  section  8.)  Chris  added  a  bunch  of 
texts  from  prior  documents  to  the  end  of  section  5  and  in 
section  6.  Just  raw  text  swatches,  not  integrated  or  edited. 
Certainly  internally  redundant,  and  maybe  redundant  with 
stuff  in  section  5. 

.7 

1 1/24/14 

Sonja  made  edits  to  section  6.  And  checked  cross-references 
and  reference  doubles. 

.8-fin 

1/12/2014 

Final  edits  to  document. 

2.  Overview  and  scope 

The  goal  of  this  document  is  capture  how  to  represent  non-  or  extra-verbal  “body 
behaviors  (e.g.,  posture,  movement,  gesture,  facial  expressions,  verbal  prosody,  tone  and 
volume,  etc.)  within  the  framework  SIFT  initially  developed  for  verbal  interactions  in  our 
Etiquette  Engine™,  initially  developed  as  an  adjunct  to  DARPA’s  DARWARS  projects. 

3.  Background  and  Outline 

The  Etiquette  Engine  (which  will  be  described  in  more  depth  below)  was  a  computational 
instantiation  and  elaboration  of  Brown  and  Levinson’s  [1]  theoretical  model  of  politeness 
and  its  role  in  shaping  and  managing  interpersonal  relationships.  It  was  based  primarily 
on  verbal  expressions  in  discourse  and  relied  primarily  on  word  usage  in  utterances  to 
trigger  and  compute  effects  in  the  turn  taking  typical  of  dialogue.  The  goal  of  this  work 
is  to  explore  how  the  Etiquette  Engine  might  be  expanded,  both  in  terms  of  its  underlying 
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theory  and  in  terms  of  its  computational  representation,  to  incorporate  non-  and  extra¬ 
verbal  behaviors.  We  will  collectively  refer  to  these  as  “body  behaviors”. 

We  expect  core  challenges  to  include: 

1.  The  need  to  address  the  continuous  nature  of  body  behaviors — Most  verbal 
behaviors  are  largely  discretized  in  time  by  the  turn  taking  nature  of  discourse. 
That  is,  one  person  speaks  and  that  utterance  contains  linguistic  tokens,  and 
finishes  largely  before  another  person  begins.  Thus,  verbal  politeness  behaviors 
have  a  more  nearly  fixed  point  and  duration  in  time.  Many  body  behaviors 
(especially  fully  non-verbal  ones)  can  persist  over  multiple  utterances,  and  can  be 
maintained  during  the  Other’s  “turn”. 

2.  The  need  to  provide  cross-cultural  scores  for  the  “redressive  value”  of  body 
behaviors — As  will  be  discussed  in  more  detail  below,  the  Etiquette  Engine  (EE) 
operated  by  detecting  known  verbal  politeness  behaviors  from  a  pre -defined, 
culture-specific  data  base  which  also  included  an  expected  “redressive  value”  for 
that  behavior — a  representation  of  its  “potency”  to  give  or  ameliorate  offense 
within  the  culture.  While  defining  and  scoring  such  behaviors  for  the  data  bases 
was  a  non-trivial  task,  we  were  greatly  facilitated  in  that  task  by  the  prior  work  of 
Brown  and  Levinson.  They  had  developed  what  amounts  to  a  taxonomy  of 
culturally  universal  linguistic  redressive  behaviors,  divided  into  a  few  larger 
“families”  and  had  suggested  a  culturally  universal  redressive  value  for  each 
family  or  type.  While  this  left  the  task  of  discovering  and  representing 
characteristic  ways  of  realizing  those  universal  types  in  each  culture  we  created  a 
data  base  so,  that  work  could  progress  in  accordance  with  the  taxonomic 
structure  Brown  and  Levinson  provided.  No  such  structure  exists  (to  the  best  of 
our  knowledge)  for  body  behaviors.  Thus,  we  will  face  the  challenge  of 
suggesting  one — either  by  reference  to  Brown  and  Levinson’s  structure  for 
verbal  behaviors  or  through  some  other  structure  we  identify  or  create. 

3.  The  need  to  incorporate  context — An  initial  perception  is  that  interpretation  of 
body  behaviors  are  far  more  determined  by  context  than  that  of  linguistic 
behaviors.  A  slouch  may  be  indicative  of  lack  of  respect  unless  it  is  being 
performed  by  an  individual  carrying  a  heavy  load,  or  wounded,  etc.  By  contrast, 
the  presence  or  absence  of  a  “sir”  (as  in  “yes,  sir!”)  is  less  subject  to  modification 
by  contextual  factors — though  it  is  not  immune  to  them. 

We  are  not,  in  this  work,  attempting  to  solve  the  problem  of  recognizing  or  generating 
body  behaviors  in  real  human  behaviors  or  avatars.  Similarly,  we  are  not  tackling  the 
overall  problem  of  how  the  detection  of  a  behavior  might  affect  the  attitudes  or  decision¬ 
making  or  an  agent,  or  why  the  agent  might  want  to  produce  a  specific  body  behavior  at  a 
given  time.  Instead,  as  with  our  earlier  EE,  we  are  thinking  toward  a  computational 
module  that  can  take  such  behaviors  as  input  from  another  system,  reason  about  and 
report  on  their  perceived  threat  or  reward — and  do  similar  reasoning  about  a  set  of 
candidate  behaviors  that  might  be  produced  in  response. 

The  rest  of  this  document  is  structured  as  follows: 
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•  First,  in  section  4.  we  introduce  the  Brown  and  Levinson  Politeness  model  and 
theory,  since  it  underpins  much  of  our  prior  thinking  and  sets  the  stage  for  what 
we  will  be  working  toward  here. 

•  Next,  in  section  5,  we  describe  SIFT’s  prior  implementation  of  the  Brown  and 
Levinson  model  in  a  computational  approach  to  scoring  perceived  politeness  of, 
primarily,  utterances  in  the  Etiquette  Engine™. 

•  In  section  6,  we  describe  two  prior  implementations  of  the  Etiquette  Engine  we 
have  made  within  a  broader  simulation  or  software  context:  These  efforts 
provide  some  guidance  as  to  how  to  fold  the  etiquette  and  face-based  reasoning 
of  the  EE  into  both  a  broader  simulation  environment  and  a  broader  model  of 
agent  goals,  etc.  Thus,  they  may  provide  useful  background  for  achieving  similar 
integration  of  reasoning  about  body  behaviors. 

•  Next,  in  section  7,  we  will  review  literature  on  cross  cultural  body  behaviors  and 
their  relevance  to  the  kinds  of  dimensions  represented  in  our  EE. 

•  In  section  8,  we  will  propose  methods  for  altering  the  core  representation  used  in 
the  EE  to  incorporate  body  behaviors. 

•  Finally,  in  section  9,  we  will  work  an  example  illustrating  how  body  behaviors 
may  be  represented  and  reasoned  about  using  the  modified  EE  approach. 

•  Section  10  provides  brief  conclusions  and  summarizes  our  results. 

4.  The  Brown  and  Levinson  Model  of  Politeness 

Brown  and  Levinson  [1],  through  extensive  cross  cultural  socio-linguistic  analysis, 
developed  a  “functional”  model  of  the  role  of  politeness  in  all  human  discourse.  The 
model  assumes  that  social  actors  are  motivated  by  a  set  of  wants  including  important 
social  wants  based  on  the  concept  of  “face”  [2]  or,  loosely,  the  “positive  social  value  a 
person  effectively  claims  for  himself’  (cf.  Cassell  and  Bickmore,  2003,  p.  6,  [3]).  Face 
can  be  “saved”  or  lost,  and  it  can  be  threatened  or  conserved  in  social  interactions. 
Virtually  all  interactions  between  social  agents  involve  some  degree  of  threat  to  the 
participants’  face — what  Brown  and  Levinson  call  Face  Threatening  Acts  (FT As).  My 
simple  act  of  speaking  to  you,  regardless  of  the  content  of  my  words,  places  a  demand  on 
your  attention  that  threatens  your  face.  If  I  simply  state  my  desire  that  you  give  me  the 
salt  as  bald  propositional  content  (e.g.,  “Give  me  the  salt”)  I  may  efficiently  communicate 
my  desire,  but  I  have  also  been  ambiguous  about  whether  or  not  I  have  the  power  or  right 
or  can  otherwise  compel  you  to  give  me  salt.  You  might  well  take  offense  at  the 
implication  that  I  could  demand  salt  from  you,  and  in  this  way,  I  would  have  threatened 
your  face. 

The  “please”  in  the  example  above  is  an  example  of  a  politeness  strategy  used  to 
“redress”  or  mitigate  the  threat  or  imposition  contained  in  the  request.  Furthermore,  the 
expectation  that  such  a  strategy  be  used  in  certain  contexts  is  an  example  of  etiquette  that 
enables  interpretations.  The  etiquette  which  we  believe  to  be  in  play  entitles  us  to 
conclude  that  those  who  use  “please”  in  an  appropriate  context  are  striving  to  play  by  the 
rules — striving  to  be  seen  as  polite;  those  who  do  not  are  not  striving  to  be  polite  for 
various  reasons  (perhaps  they  don’t  believe  they  need  to  be,  perhaps  their  notions  about 
politeness  are  different,  perhaps  they  are  just  rude.) 
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The  core  of  the  model  is  the  claim  that  the  face  threat  posed  by  an  act  is  provided  by  the 
function: 


1/14=  D(S,H)  +  P(H,S)  +  Rx 

•  Wx  is  the  ‘weightiness’  or  severity  of  the  FTA 

•  D(S,H)  is  the  social  distance  between  the  speaker  (S)  and  the  hearer  (H).  Social 
distance  is  roughly  equivalent  to  familiarity — it  increases  with  contact  and 
interaction,  but  may  also  with  be  based  on  a  priori  factors  such  as  membership  in 
the  same  family,  clan  or  organization. 

•  P(H,S)  is  the  relative  power  that  H  has  over  S.  Power  comes  from  different 
sources  in  different  cultures  and  organizations.  Non-player  characters  (NPCs) 
representing  commanders,  subordinates,  or  high  or  low  status  citizens  might  all 
need  to  act,  and  to  be  handled  according  to  different  etiquettes  if  face  threats  are 
to  be  minimized. 

•  Rx  is  the  ranked  imposition  of  the  raw  act  itself.  Some  degree  of  imposition  is 
culturally  defined — it  may  be  inherently  more  of  an  imposition  to  request  food 
from  a  host  in  Western  culture  than  in  an  Arabic  one,  for  example.  But 
imposition  is  also  dependent  upon  the  roles  and  duties  of  the  parties  involved. 

Since  FTAs  are  potentially  disruptive  to 
human-human  relationships,  and  since  we 
generally  wish  to  avoid  disruption,  we 
make  use  of  redressive  strategies  to 
mitigate  the  degree  of  face  threat  imposed 
by  our  actions.  Brown  and  Levinson  offer 
an  extensive  catalogue  of  universal 
strategies  for  redress,  organized  according 
to  5  broad  strategies.  These  are  illustrated 
in  Figure  1  ranked  from  least  to  most 
threatening.  The  least  threatening 
approach,  obviously,  is  simply  not  to  do 
the  FTA,  and  this  is  sometimes  the  only 
acceptable  strategy  in  a  cultural  context.  If  one  is  to  do  the  FTA  at  all,  then  the  least 
threatening  way  to  do  it  is  “off  record”.  Off  record  FTA  strategies  are  means  of  doing  the 
act  with  a  sort  of  “plausible  deniability”  by  means  of  innuendo  and  hints.  If  one  does  the 
FTA  overtly,  then  one  can  still  undercut  its  degree  of  threat  by  offering  redress  aimed  at 
either  positive  or  negative  face.  Negative  redressive  strategies  focus  on  H’s  needs  for 
independence  of  action  and  attention.  They  minimize  the  impact  on  H  by  being  direct 
and  simple  in  making  the  request,  offering  apologies  and  deference,  minimizing  the 
magnitude  of  the  imposition  and/or  explicitly  incurring  a  debt.  Positive  redressive 
strategies  target  the  hearer’s  desire  that  his/her  needs  and  wants  be  seen  as  desirable. 
These  strategies  emphasize  common  ground  between  S  and  H  by  noticing  and  attending 
to  H,  by  invoking  in-group  identity,  by  joking  and  assuming  agreement  and/or  by 
explicitly  offering  rewards/promises.  Finally,  the  most  threatening  way  of  performing  an 
FTA  is  “baldly,  on  record”  and  without  any  form  of  redress. 

Brown  and  Levinson’s  model  doesn’t  stop  at  that  level,  however.  For  positive  and 
negative  redressive  and  off  record  strategies,  they  offer  a  host  of  well-researched 


/  On  record 


Do  the  FTA  ( 


w/  Redress 


1.  w/o  Redress, 

baldly 

2.  Positive 
Politeness 

v  3.  Negative 
Politeness 


4.  Off  record 


'  5.  Don’t  do  the  FTA 


Figure  1 .  Universal  FTA  Ftedress  strategies  ranked  by 
Brown  and  Levinson,  (1987). 
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Redress  other 
wants  of  H, 


2.  Question,  hedge 


1.  Be  conventionally  indirect 


3.  Be  pessimistic 


pronouns  I  and  you 

8.  State  the  FTA  as  a  general 
rule 

9.  Nominalize 


5.  Give  deference 


6.  Apologize 

7.  Impersonalize:  avoid 


4.  Minimize  the  imposition,  Rx 


10.  Go  on  record  as  incurring 
a  debt,  or  as  not  indebting 
H 


derivative  from 
negative  face 


Figure  2.  Specific  negative  politeness  strategies  identified  by  Brown  and  Levinson  (1987),  their  rationale  and 
relationships. 


examples  from  at  least  three  different  language/culture  groups  (English,  Tamil  and 
Tzeltal)  organized  and  abstracted  into  a  universal  structure  of  mutually  supporting  and 
incompatible  approaches.  Their  organizational  structure  for  ten  different  negative  redress 
strategies  is  presented  in  Figure  2  as  an  example.  Note  again  that  these  are  abstract, 
culture -independent  strategies  whose  relative  utility  is  captured  by  the  model  in  Figure  1, 
though  they  may  well  need  a  culture-specific  instantiation  prior  to  use — for  example,  the 
specific  behaviors  that  count  as  an  “apology”  may  well  differ  from  culture  to  culture,  but 
the  basic  utility  of  using  an  apology  will  hold  across  cultures. 


5.  SIFT’s  Implementation  of  Brown  and  Levinson  in  the 
Etiquette  Engine™ 


5. 1  Our  Modifications  and  Use  of  the  Model 

Brown  and  Levinson’s  work  provides  us  with  a  rich,  universal  model  of  how  and  why 
specific  acts  are  seen  as  polite  or  impolite,  as  well  as  abstract,  culturally  universal 
strategies  for  steering  the  perception  of  an  act  toward  either  politeness  or  impoliteness. 
Furthermore,  it  provides  us  with  variables  of  social  interaction  (D,  P,  and  R)  that  serve  to 
either  mandate  a  specific  level  of  politeness,  or  about  which  inferences  can  be  made 
given  an  observed  level  of  politeness.  In  2004-2007,  we  were  engaged  in  a  pair  of 
DARPA-sponsored  projects  under  the  direction  of  Dr.  Ralph  Chatham  to  quantify  this 
model,  make  it  computational  and  to  evaluate  its  predictive  power.  Our  vision, 
ultimately,  was  to  use  this  model  for  generating  adaptive  social  etiquette  behaviors  in  a 
Non-Player  Character  (NPC)  of  a  given  cultural  background  in  a  given  context  (where 
context  can  include  variable  aspects  of  familiarity,  power  relationships  and  imposition — 
the  D,  P  and  R  described  above). 
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In  the  earlier  of  these  two  projects,  we  refocused  the  Brown  and  Levinson  model  on 
making  predictions  about  how  believable  the  behaviors  of  an  NPC  will  appear  to  an 
observer.  To  do  this,  we  implemented  a  quantified,  computational  version  of  the  Brown 
and  Levinson  equation  that  used  perceived  aspects  of  context  (D,  P  and  R,  as  well  as 
known  history  about  the  character,  C,  of  the  actors)  to  predict  an  expected  level  of 
redressive  behaviors  from  an  NPC  actor.  NPCs  which  do  not  exhibit  the  expected  degree 
of  polite  redress  (either  by  being  over  or  under  polite)  were  expected  to  be  seen  as  either 
unbelievable  or  to  invite  rethinking  of  what  was  previously  understood  about  the  D,P,R 
and  C  of  the  context.  For  example,  if  a  private  bursts  in  on  his  captain  and  screams  out  a 
warning  without  a  salute,  an  observer  might  well  assume  that  the  degree  of  imposition 
(R)  is  less  than  might  otherwise  be  the  case  because  the  private  was  charged  with 
providing  such  warnings  or  expects  it  to  be  in  the  captain’s  best  interests  to  hear. 

During  this  prior  research,  we  developed  specific  scenarios  which  were  pertinent  to  the 
gaming  environment  under  development  by  Dr.  Lewis  Johnson  at  the  Center  for 
Advanced  Research  in  Technology  for  Education  (CARTE)  at  the  University  of  Southern 
California.  The  gaming  environment  was  the  Tactical  Language  Training  System 
(TLTS)  sponsored  by  the  DARPA  DARWARs  project  as  a  game -based  approach  to 
simple  foreign  language  training  (see  also  section  7  for  more  details).  While  makes 
extensive  use  of  a  graphical  environment  and  of  speech  recognition  and  speech 
generation,  we  believed  that  it  did  not  contain  a  generative  model  of  social  interactions  or 
politeness  (though  this  turned  out  to  be  only  partly  true).  Our  test  scenarios  involved 
specific  and  controlled  manipulations  of  the  politeness  variables  P,  D,  R  and  C  and 
specific  redressive  behaviors  exhibited  by  the  game’s  NPC  characters.  Our  scenarios 
took  the  form  of  written  texts  and  they  were  designed  (according  to  our  expectations)  to 
embody  specific  predictions  from  the  Brown  and  Levinson  model: 

•  High  Face  Threat  and  High  Redress — which  should  be  balanced  and  yield 
nominal  (believable)  politeness 

•  High  Face  Threat  and  Low  Redress,  which  should  result  in  unredressed  threat 
and  therefore  be  perceived  as  rude 

•  Low  Face  Threat  and  High  Redress,  which  should  result  in  more  redress  than  is 
needed  to  redress  the  face  threat  and,  therefore,  be  perceived  as  over-polite. 

•  Low  Face  Threat  and  Low  Redress,  which  should  also  be  balanced  and  be 
perceived  as  nominally  polite  and,  therefore,  believable. 

Two  versions  of  these  scenarios  were  scored  using  our  quantitative  etiquette  algorithm. 
The  results  were  shown  to  track  the  predicted  design  of  the  scenarios  exactly:  Scenarios 
that  were  expected  to  exhibit  nominal,  believable  politeness  levels  achieved  low  scores, 
all  near  zero.  Scenarios  that  were  expected  to  exhibit  excessive  politeness  achieved  high 
scores,  all  substantially  larger  than  100.  Scenarios  expected  to  be  seen  as  rude  all 
achieved  very  low  scores,  substantially  smaller  than  -100  (cf.  [4]).  This  scoring  example 
will  be  presented  in  more  detail  below,  as  it  will  serve  as  the  basis  of  our  novel  work  on 
this  project. 

Thus,  the  specific  product  of  our  SBIR  was  a  partially-validated  computational  approach 
to  scoring  the  imbalance  of  politeness  usages  in  a  social  interaction  context  and, 
therefore,  of  computing  the  degree  to  which  a  given  utterance  would  be  perceived  as 
polite,  rude  or  nominal  in  context — that  is,  by  a  given  observer,  potentially  from  a 
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specific  culture,  observing  the  utterance  being  said  by  a  given  speaker  to  a  given  hearer, 
all  with  implications  for  the  various  terms  in  the  model  described  above.  This  module 
could  be  used  to  score  observed  behavior  or,  through  evaluating  alternate  proposed 
behaviors,  to  inform  the  selection  of  desired  behaviors  for  different  goals  or  ends.  This 
means  that  it  could  be  used,  as  a  general,  reusable,  computational  algorithm,  to  equip  an 
NPC  with  the  ability  to  detect  when  they  are  being  addressed  with  more  or  less  redressive 
behaviors  than  they  should  expect  given  what  they  know  about  P,  D  and  R  within  their 
social  context,  and  therefore,  to  reactively  take  offense  or  take  advantage.  Similarly,  it 
could  be  used  to  equip  NPCs  with  the  ability  (again,  in  a  reusable,  algorithmic  form)  to 
determine  appropriate  behaviors  for  them  to  exhibit  in  order  to  further  their  ends.  In  a 
game-based  training  context,  this  means  precisely  that  the  NPCs  can  make  use  of  this 
reusable  etiquette  module  to  behave  believably  in  their  social  interaction  with  human 
players.  If  the  player  is  offensive,  the  NPC  will  take  offense  and  will  react  either  by 
ascribing  to  the  player’s  actions  more  power  (P),  less  familiarity  (D)  or  lower  imposition 
(R)  and  therefore,  may  react  by  providing  more  respectful  behaviors  (i.e.,  more  redress) 
in  return,  or  by  asserting  increased  power  and  less  familiarity  in  response — and 
decreasing  redressive  behaviors.  Alternatively,  if  the  player  is  polite,  the  NPC  may 
assume  s/he  is  claiming  less  power,  more  familiarity  or  a  greater  imposition,  and  may 
react  with  more  cooperation  and  politeness  or  may  assume  superiority  over  the  player. 

The  use  of  Brown  and  Levinson’s  model  and  theory  to  inform  a  module  for  reasoning 
about  social  interaction  behaviors  guarantees  (insofar  as  Brown  and  Levinson’s  work  is 
correct)  that  the  module  will  be  universal  in  its  reasoning  about  and  scoring  of  abstract 
politeness  “moves”.  As  we  have  noted  above,  however,  any  such  module  will  need  to  be 
equipped  with  culture-specific  knowledge  bases  to  enable  reasoning  from  the  observable 
behaviors  in  a  culture  (e.g.,  pursed  lips  or  a  rigid  hand-to-eyebrow  salute)  to  the  abstract 
etiquette  “moves”  (and  therefore,  politeness  implications)  over  which  the  model’s 
parameters  are  scored.  This  has  the  practical  implication  that  the  general  social 
interaction  reasoning  of  an  NPC  within  a  game  context  can  be  effectively  modularized 
(as  illustrated  in  Figure  3)  and,  thus,  large  savings  in  game  code  development  can  be 
realized  and/or  NPCs  can  be  developed  to  exhibit  a  larger  and  more  sophisticated  range 
of  social  interaction  behaviors  (as  will  be  discussed  in  section  6.3  below). 

5.2  “Belie  vable  ”  Le  vels  of  Politeness 

According  to  the  Brown  and  Levinson  model  described  above,  people  generally  want  to 
accomplish  their  goals  expeditiously—  and  this  argues  for  minimizing  redressive 
strategies.  But  they  also  experience  a  range  of  social  and  personal  pressures  to  not 
threaten  the  face  of  those  they  interact  with  (especially  those  with  greater  power  or  shared 
familiarity)—  and  this  argues  for  extensive  redressive  strategies.  The  balance  between 
these  pressures  yields  the  selection  of  specific  strategies  in  context.  Brown  and  Levinson 
allude  to,  but  don’t  explicitly  include  a  factor  representing  the  relative  weighting  that  an 
individual  puts  on  his/her  own  goals  vs.  the  face  goals  of  others—  his/her  general 
willingness  (independent  of  the  other  factors)  to  place  others’  needs  first.  For  want  of  a 
better  term,  we’ve  called  this  factor  “character”  and  introduced  a  term  for  it,  abbreviated 
as  C,  with  the  character  of  speaker  (S)  being  C(S).  In  other  words,  the  degree  of  redress 
that  a  speaker  S  chooses  to  use  will  be  a  function  of  the  degree  of  face  threat  inherent  in 
the  act  (itself  a  function  of  P,D  and  R)  and  the  speaker’s  character  C(S). 
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But  the  above  description,  and 
indeed  Brown  and  Levinson’s 
primary  focus,  is  from  the 
perspective  of  the  speaker/actor 
(S)  interested  in  achieving 
interaction  goals  and, 
presumably,  in  avoiding  face 
threat  to  hearers  (Hs).  We  can 
characterize  Brown  and 
Levinson’s  model  graphically 
as  in  the  top  portion  of  Figure  3. 

A  Speaker  with  a  given 
character  C(S),  uses  his/her 
knowledge  of  the  D,  P,  and  R  of 
a  given  context  and  desired 
FTA  in  order  to  select  one  or 
more  strategies  from  among  a 
knowledge  base  of  culture-specific  strategies  resulting  in  a  specific  action  Ax  which  is 
designed  to  both  further  S’s  goals  and  to  avoid  undue  face  threat  to  his/her  interlocutors. 

In  order  to  implement  and  make  use  of  this  model  in  believable  human-computer 
interactions  (i.e.,  with  simulated  agents),  we  needed  to  take  the  perspective  of  an 
observer/hearer  (who  may  or  may  not  be  the  one  the  speaker  is  actually  interacting  with). 
This  perspective  is  represented  graphically  at  the  bottom  of  Figure  3.  Here,  an  observer 
(O)  perceives  an  utterance  that  has  bald  content  as  a  speech  act  and  may  or  may  not 
contain  culturally-recognized  redressive  strategies.  O  also  has  access  to  additional  cues 
from  his/her  perception  of  the  context  and  perhaps  memory  for  past  history.  Given  these 
cues,  O's  goal  is  to  construct  a  picture  of  the  "politeness"  character  of  S  and,  through  that, 
to  the  P,  D  and  R  of  the  interaction  between  S  &  H.  Fundamental  to  our  approach  is  the 
claim  that  this  construction  process  is  based  largely  on  the  degree  of  match  or  mismatch 
between  the  redressive  strategies  actually  used  by  S  (as  perceived  by  O)  and  those 
expected  by  O. 

Given  his/her  own  observations  or  knowledge  of  the  context,  O  can  construct  an 
understanding  of  the  parameters  P,  D,  and  R.  For  example,  if  S  is  noticeably  older, 
richer,  or  is  wearing  insignia  that  make  it  clear  that  s/he  outranks  H,  then  O  might 
reasonably  conclude  that  the  power  distance  (P)  between  them  is  large  and  favors  S. 

Note  that  the  factors  that  affect  O’s  interpretation  of  power  will  themselves  be  dependent 
both  on  O’s  noticing  them,  and  on  O’s  culture-based  knowledge  and  interpretation  of 
those  triggers.  If  S  and  H  are  behaving  familiarly  (standing  close  together,  interacting 
jovially,  using  nicknames,  etc.),  are  known  to  be  related  as  family  members  or  friends, 
etc.,  then  O  might  conclude  that  the  social  distance  (D)  is  comparatively  small  between 
them.  Finally,  O  will  have  his/her  own  culturally-based  beliefs  about  the  degree  of 
imposition  (R)  of  a  given  act  (e.g.,  asking  for  money  is  a  greater  imposition  than  asking 
for  help  finding  a  location,  which  is  a  greater  imposition  than  asking  for  the  time),  but 
observed  or  known  characteristics  of  the  interaction  may  also  serve  to  reduce  the 
perceived  R.  For  example,  if  S  is  known  to  have  a  duty  (perhaps  based  on  his/her  role)  or 
a  standing  request  to  provide  certain  information  or  advice  to  H,  or  if  H  is  not  apparently 
engaged  in  any  ongoing  activity. 
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R  ^ 

Culture-specific 
redressive 
_  strategies  _ 

Culture-specific 

Markers  J 

Action  Production  in  our  modification  and  application 


Contextx  Assumed  Expected 

/  D(S,H),  P(H,S),\  Redress  in 
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Unbelievable" 
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Rgure  3.  Graphical  depiction  of  the  Brown  and  Levinson 
model  and  our  proposed  modifications  to  it. 
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Then,  given  his/her  beliefs  about  these  parameters,  O  can  construct  an  estimate  of  the 
degree  of  face  threat  associated  with  the  bald  content  of  the  act.  Furthermore,  given 
whatever  information  s/he  possess  about  C(S),  O  can  adjust  his/her  predictions  about  the 
degree,  and  therefore  the  types,  of  redressive  actions  that  s/he  might  expect  to  see  used. 
Let  us  call  this  product  the  expected  redressive  act  (Ax). 

At  the  same  time,  O  can  actually  perceive  an  observed  redressive  act  (Ax).  S  performs  an 
act  that  O  will  perceive  as  having  a  degree  of  imposition  and,  perhaps,  various  associated 
redressive  actions.  If  the  observed  act  and  the  expected  act  are  the  same  (perhaps  within 
certain  degrees  of  tolerance),  then  the  actor  will  be  seen  as  believable — at  least  with 
regards  to  his/her/its  politeness-producing  etiquette  behaviors. 

Therefore,  one  metric  for  believability  is  the  delta  between  the  expected  act  and  the 
perceived  act.  And  yet,  other  humans  fail  to  behave  as  we  expect  them  to  behave  all  the 
time  without  our  labeling  them  “unbelievable”.  This  seems  to  be  because  humans  are 
generally  aware  that  predicting  politeness  behaviors  is  far  from  an  exact  science.  We  are 
generally  more  willing  to  revise  our  beliefs  about  aspects  of  the  context  or  character  that 
produced  our  initial  predictions  and  then  reassess  that  prediction  than  we  are  to  conclude 
that  S  is  acting  artificially.  Thus,  assessments  are  likely  influenced  by  (cognitive)  biases 
and  (cultural)  stereotypes. 

This  metric  may  be  computed  over  time  as  well.  If  successive  actions,  with  their 
associated  degrees  of  redress  employed,  continue  to  violate  O’s  notions  of  the  NPCs’ 
context  and  P,D,R  and  C  values,  O  may  choose  to  revise  the  assumed  characteristics 
seeking  a  set  of  P,D,R  and  C  values  that  minimizes  the  delta  between  expected  and 
observed  degrees  of  redress.  If  no  such  model  is  found,  or  if  violations  are  extreme,  s/he 
may  give  the  game  up  and  simply  declare  those  behaviors  to  be  “unbelievable”. 

We  have  not  conducted  extensive  studies  of  the  quantitative  predictions  of  this  model. 
Therefore,  knowing  how  long  it  takes  observers  to  revise  their  models  is  an  empirical 
question  that  we  cannot  currently  address.  It  seems  quite  reasonable,  however,  that  the 
more  data  that  the  observer  has  amassed  to  form  impressions  of  P,D,R  and  C,  the  more  it 
will  take  to  provoke  a  revision  in  those  beliefs.  In  cases  where  no  prior  data  has  been 
amassed,  we  assume  that  observers  will  resort  to  presumed  culturally-defined  norms  or 
averages  (e.g.,  If  I’m  told  only  that  “Person  A”  is  talking  to  “Person  B”,  I  am  likely  to 
assume  that  both  are  of  average/nominal  power,  neutral  social  distance,  etc.).  Note  that, 
though  extensive  naturalistic  analyses  have  not  been  performed  by  us,  in  one  detailed 
analysis  [5]  this  model  was  able  to  explain  a  set  of  naturalistic  observations  of 
sequentially  occurring  cross-cultural  tensions  between  U.S.  and  Russian  crew  members 
on  board  the  International  Space  Station. 

5.3  The  “Etiquette  Algorithm” 

We  implemented  a  version  of  the  Brown  and  Levinson  equation  to  use  as  a  predictive 
model  of  the  believability  of  the  redressive  actions  of  a  computerized  game  character  (a 
Non  Player  Character  or  NPC)  as  it  appears  to  a  human  observer,  with  perceived  aspects 
of  context  (D,  P  and  R,  as  well  as  known  history  about  the  character,  C,  of  the  actors). 
NPCs  which  do  not  exhibit  the  expected  degree  of  polite  redress  (either  by  being  over-  or 
under-polite)  were  expected  to  be  seen  as  either  unbelievable  or  to  invite  rethinking  of 
what  was  previously  understood  about  the  D,P,R  and  C  of  the  context.  For  example,  if  a 
private  bursts  in  on  his  captain  and  issues  a  bald  directive  (“Get  your  coat  on”)  without 
any  redress,  an  observer  might  well  assume  that  the  degree  of  imposition  (R)  is  less  than 
might  otherwise  be  the  case  because  the  private  was  charged  with  giving  such 
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instructions,  or  that  the  familiarity  between  them  warranted  it.  Otherwise  (and  especially 
in  a  simulated  environment),  the  observer  might  simply  believe  that  the  private  was 
behaving  “unbelievably”. 

Expanding  on  the  Brown  and  Levinson  equation  for  Face  Threat,  as  presented  above,  our 
implementation  uses  weights  on  each  component  to  allow  the  possibility  to  value  D,  P, 
and  R  differently  (a  factor  we  suspect  may  underlie  some  cultural  differences),  resulting 
in  the  equation  below: 

Wx  =  [wrD(S.H)]  +  [w2-P(H,S)]  +  [w3-Rx]  +  C(S) 

Each  Observer  adds  his/her  own  interpretations  of  the  context.  For  example,  D(S,H) 
could  be  expanded  to  [BH:wf  Bh:D(S,H)],  representing  Hearer’s  belief  about  the  degree 
of  social  distance  and  the  Hearer’s  belief  about  the  appropriate  weight  for  the  social 
distance  term.  We  use  Speaker  belief  (Bs:)  and  Observer  (who  could  also  be  a  Speaker  or 
Hearer)  belief  (B0:)  similarly.  This  results  in  the  following  expansion  for  an  Observer  O: 


B0:WX  =  {[Boiwr  B0:D(S,H)]  +  [B0:w2-  B0:P(H,S)]  +  [B0:w3-  B0:RX]}  +  B0:C(S) 

An  implication  of  the  Brown  and  Levinson  model,  though  never  overtly  spelled  out,  is 
that  the  weightiness  of  the  face  threat  must  be  fully  compensated  for,  or  “redressed”  in 
normal  interactions  if  the  status  quo  in  relationships  is  to  be  maintained.  Therefore,  the 
value  of  Wx  should  be  balanced  by  the  “value”  (V)  of  a  set  of  redressive  actions  used  in 
the  interaction  x  (Ax)  if  the  interaction  is  to  appear  “normal”  or  believable  or  without 
ulterior  motive.  In  other  words,  we  expect  the  value  of  the  redressive  strategies  the 
speaker  uses  to  equal  or  balance  the  value  of  the  face  threat  s/he  produces,  or: 

Wx  =  V(AX) 

This  means  that  an  Observer’s  beliefs  and  weightings  of  the  social  distance,  power  and 
imposition  relationships,  adjusted  by  belief  about  the  character  of  the  Speaker,  should  be 
balanced  by  the  Observer’s  belief  about  the  value  of  the  set  of  redressive  behaviors  used. 
We  express  this  as  a  difference  to  give  us  an  “incredibility”  or  “imbalance”  metric  which 
also  serves  as  a  perceived  politeness  metric: 

B0:lx=  B0:V(Ax)  -  B0:WX 

In  order  to  use  this  metric  to  evaluate  the  imbalance  between  expected  and  observed 
levels  of  politeness,  we  must  operationalize  the  various  parameters.  We  will  summarize 
the  basic  approach  to  doing  so  below. 

5.4  Operationalizing  EQ  Terms 

In  order  to  operationalize  and  quantify  the  Brown  and  Levinson  model  described  above, 
we  first  had  to  develop  scalar  values  for  the  various  politeness  parameters  P,D,  R  and  C. 
We  proposed  that  the  variables  D(S,H),  P(H,S)  and  Rx,  as  well  as  the  various  parties’ 
perceptions  of  them,  be  represented  as  continuous  scalar  values  ranging  from  negative  to 
positive  infinity.  The  value  of  0  is  the  “balance  point”  or  a  nominal  or  equal  value  for 
each  scale,  while  positive  values  indicate  that  the  dimension  is  increased  (and  contributes 
to  an  increasingly  “weighty”  or  potent  FT  A)  and  negative  values  indicate  that  it  is 
decreased  (and  is,  in  fact,  building  up  the  Hearer  rather  than  threatening  him  or  her).  For 
Power  Difference  of  the  Hearer  over  the  Speaker  (P(H,S)),  for  example,  a  value  of  0 
means  that  the  power  of  the  Hearer  and  the  Speaker  are  equal,  that  they  are  (exact)  peers. 
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Values  greater  than  0  indicate  that  the  Hearer  (H)  has  increasingly  greater  power  (as 
values  increase)  relative  to  S  and,  therefore,  that  face  threat  increases  whenever  S 
addresses  H.  We  proposed  the  following  scale  anchor  points  for  nominal  American 
culture: 

•  A  value  of  -1000  is  characteristic  of  the  power  that  a  CEO  of  a  major  company 
(as  S)  has  “over”  (or  relative  to)  a  janitor  in  his/her  company  (as  H)  or  the  power 
that  a  parent  has  over  a  small  child. 

•  A  value  of  -100  is  characteristic  of  the  power  that  a  professor  has  relative  to  a 
freshman  student  or  a  parent  over  an  early  teenage  child. 

•  A  value  of  -10  is  characteristic  of  the  power  that  a  project  manager  in  an  informal 
research  team  has  over  project  members,  or  the  power  that  a  parent  has  over  an 
older  teenager. 

•  A  value  of  0  is  characteristic  of  equal  power  between  S  and  H;  no  or  negligible 
difference — for  example,  the  power  relationship  between  two  co-workers  at  the 
same  level  and  seniority. 

•  A  value  of  10  is  characteristic  of  the  inverse  of  the  power  described  for  -10 
above — the  power  that  an  older  teenager  or  work  team  project  member  as  S 
would  have  over  (or  relative  to)  a  parent  or  project  manage,  as  H,  respectively. 

•  Etc. 


Similar  scales  were  developed  for  D(S,H)  and  Rx.  The  character  term  (C)  was 
represented  as  a  simple  value  added  or  subtracted  from  the  Wx  sum. 

The  assigning  of  a  specific  value  within  these  scales  was  something  of  an  “best 
judgment”  by  an  engineer,  supported  through  discussions  with  a  subject  matter  expert  in 
the  language  and  culture.  We  have  not  yet  had  the  time  or  opportunity  to  collect 
sufficient  data  in  any  specific  culture  or  context  to  refine  the  scales  appropriately.  The 
fact  that  they  work  as  well  as  they  do  currently  is  a  testament  to  the  robustness  of  the 
model. 

Next,  we  developed  numerical  valuations  for  various  redressive  behaviors  based  on  the 
guidelines  provided  by  Brown  and  Levinson  as  depicted  in  Figure  1  (illustrating  the 
various  redressive  values  of  broad  classes  of  redressive  strategies)  above.  Ranges  of 
values  for  the  broad  classes  of  strategies  were  defined  as  follows,  with  individual 
strategies  within  each  class  being  assigned  a  value  within  the  designated  range: 

•  The  value  of  the  use  of  an  individual  positive  redressive  strategy  (see  Brown  and 
Levinson,  p.  102,  Figure  3,  for  a  list  of  such  strategies)  will  provide  from  1  to  40 
“units”  of  redress. 

•  The  value  of  the  use  of  an  individual  negative  redressive  strategy  (see  Brown  and 
Levinson,  p.  131,  Figure  4,  for  a  list  of  such  strategies)  will  provide  from  20  to 
60  units  of  redress. 


Within  the  range  defined  above,  a  specific  score  was  assigned  to  individual  instances  of 
redress  which  fell  into  the  category,  as  will  be  illustrated  below. 

The  effects  of  multiple  redressive  strategies  were  scored  as  simply  additive.  We 
understand  that  this  is  a  simplification,  and  that  the  efficacy  of  added  redressive 
behaviors  probably  falls  off,  eventually  becoming  simply  irritating.  This  means  that  the 
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value  V  of  a  set  of  N  redressive  actions  A  contained  in  interaction  x  is  given  by  the 
function: 


V(AX)  =  V(A-i)  +  V(A2)  +  V(An) 

5.5  Evaluation  Test  Cases 

This  approach  was  then  tested  in  a  series  of  sample  social  interaction  vignettes  crafted  to 
represent  (according  to  our  American  cultural  intuitions)  either  normal/balanced 
politeness,  excessive  over-politeness  or  excessive  rudeness.  Our  goal  was  to  determine  if 
the  equation  and  scoring  techniques  we  had  developed  would  track  our  intuitions  for 
scenarios  in  which  politeness  was  balanced  or  imbalanced  in  various  ways.  The  level  of 
face  threat  and  redress  were  varied  over  this  set  of  vignettes  so  that  high  face  threat 
situations  were  paired  with  high  levels  of  redress  (which  should  produce  roughly  normal, 
balanced  levels  of  redress)  as  well  as  low  levels  of  redress  (which  should  be  highly 
imbalanced  and  perceived  as  very  rude — perhaps  unbelievably  so).  Similarly,  very  low 
levels  of  face  threat  were  paired  with  very  high  levels  of  redress  (which  should  be 
perceived  as  over-polite,  perhaps  unbelievably  so)  and  with  low  levels  of  redress  (which 
should  be  perceived  as  balanced  and  fully  believable).  Examples  of  two  such  vignettes 
are  illustrated  below,  followed  by  our  reasoning  through  the  application  of  the  above 
scales  and  valuations  to  those  scenarios: 

Vignette  1 — High  Face  Threat,  High  Redress,  High  Believ ability:  A  low 
ranking  soldier  (i.e.,  a  corporal,  as  indicated  by  uniform  insignia)  walks  into  the 
Mayor's  office  and  the  Mayor  motions  him  quickly  to  a  seat.  The  soldier  takes 
off  his  hat  and  sits  down,  waiting  while  the  Mayor  continues  to  write  something. 
The  Mayor  finishes  up  writing,  puts  down  his  pen  and  looks  up  at  the  soldier 
expectantly.  The  soldier  then  says,  “Im  sorry  to  interrupt  your  work.  Mayor 
Fredrickson,  but  my  name  is  Corporal  Jones  and  I've  been  put  in  charge  of  your 
escort  to  the  event  tonight.  I  was  wondering  if  it  would  be  possible  for  you  to  let 
me  know  where  I  can  meet  your  wife  so  that  I  can  get  her  there  on  time?” 

Vignette  2 — High  Face  Threat,  Low  Redress,  Low  Believability :  As  for  vignette 
1  above  except  that  the  soldier  acts  and  speaks  differently.  Here,  he  interrupts 
the  mayor  while  he  is  speaking,  perhaps  by  putting  a  hand  on  his  shoulder,  and 
says  loudly,  “Tell  me  where  I  can  meet  your  wife?” 

Each  of  the  eight  vignettes  was  then  assessed  using  the  operational  scoring  tables  we  had 
created.  For  example,  for  the  first  vignette  the  imbalance  evaluation  proceeded  as 
follows: 


•  The  corporal  (as  S)  has  lower  power  than  the  mayor  by  a  fairly  large  degree. 

That  is,  his  “power  difference”  relative  to  the  mayor  is  fairly  large — perhaps 
slightly  larger  than  our  anchor  point  of  100,  yet  less  than  the  anchor  point  of 
1000.  We  scored  that  as  P(H,S)  =  300  (and,  since  there  were  no  cultural 
differences  or  speaker  or  observer  misperceptions  B0:P(H,S)  =  300). 

•  There  is  no  particular  familiarity  between  the  two  individuals  in  this  vignette,  but 
social  distance  is  not  extreme  either.  They  are  from  slightly  different  “cultures” 
(military  vs.  civilian  infrastructures)  and  show  no  evidence  of  prior  relationship, 
but  they  are  engaged  in  a  common  endeavor.  The  social  distance  between  them 
is  probably  only  slightly  higher  than  0.  Thus,  D(S,H)  =  3. 

•  The  imposition  of  this  request  could  be  somewhat  large.  To  ask  after  the  location 
of  one’s  wife  so  as  to  pick  her  up  is  comparatively  threatening,  though  the  fact 
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that  this  is  in  the  mayor’s  service  should  mitigate  this  imposition  (as  the  corporal 
reminds  him).  The  raw  imposition  is  a  short  answer  required  from  the  mayor, 
characteristic  of  our  level  10,  so  we  assigned  it:  Rx  =  10. 

•  Since  we  have  provided  no  reason  to  believe  that  the  character  of  the  corporal  is 
anything  other  than  nominal,  we  will  assume  that  C=0. 


This  gives  us  a  value  of  B0:WX  as  supplied  by  the  left  hand  portion  of  the  equation  above 
as: 


3  +  300  +  10  =  313 

For  the  value  of  the  redress  applied  V(AX)  we  identified  and  scored  the  set  of  redressive 
actions  in  Table  1. 

Thus,  the  imbalance  score  for  this  vignette,  as  calculated  by  the  equation,  would  be:  295 
-  313  =  -18.  Since  this  vignette  was  intended  to  convey  both  high  face  threat  and  high 
redress  and,  thus,  to  be  balanced  and  believable,  this  score  seems  to  be  about  right,  falling 
very  near  zero. 

For  the  second  vignette,  by  contrast,  we  have  a  high  degree  of  face  threat  with  virtually 
no  redressive  actions.  This  is  unexpected  and  should  be  perceived  as  very  rude.  This 
scenario  should  have  a  score  much  less  than  0  on  our  imbalance  metric — indicating  that 
there  is  substantial  unredressed  threat.  This  vignette  was  scored  as  illustrated  in  Table  2. 


Table  1.  Redressive  actions  scored  in  Vignette  1. 


Action  and  Interpretation 

Score 

1 .  The  soldier  waits  until  the  mayor  is  finished  and  invites  him  to  speak.  This  seems  to  be  a  very 
explicit  form  of  negative  politeness  (putting  the  other’s  interests  first)  and,  especially  in  this  instance 
where  the  H  was  not  actively  engaged  in  another  conversation,  seems  very  potent. 

60 

2.  The  soldier  takes  off  his  hat.  This  is  a  sign  of  deference  in  our  culture,  which  is  in  turn  a  fairly 
potent  negative  politeness  strategy. 

50 

3.  The  soldier  apologizes  for  interrupting.  This  is  also  a  negative  politeness  strategy,  though  arguably 
a  less  potent  one  (though  that  may  be  highly  mitigated  by  facial  expressions  and  body  language). 

30 

4.  The  soldier  uses  an  honorific.  Moderately  potent  negative  politeness  strategy. 

40 

5.  The  soldier  poses  the  FTA  as  a  question.  Common  negative  politeness  strategy. 

20 

6.  The  soldier  offers  an  explanation/reason  for  needing  the  information.  Positive  politeness  strategy, 
seems  particularly  powerful  in  this  case. 

35 

7.  The  soldier  appeals  to  the  Mayor’s  (H’s)  interests.  Positive  politeness  strategy  Powerful  in  this 
context. 

30 

8.  The  soldier  is  hesitant  and  skeptical  about  compliance.  A  common  but  reasonably  potent  negative 
politeness  strategy. 

30 

TOTAL 

295 
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Table  2.  Redressive  actions  scored  in  Vignette  2. 


Action  and  Interpretation 

Score 

1 .  The  soldier  is  very  brief  and  therefore,  takes  little  of  the  mayor’s  time.  This  could  be  counted  as  a 
negative  politeness  strategy  of  directness  (albeit  not  a  very  effective  or  unambiguous  one). 

30 

2.  This  could  perhaps  also  be  counted  as  an  example  of  the  positive  politeness  strategy  of  optimism 
and  assumed  compliance — though  again,  not  unambiguously. 

10 

TOTAL 

40 

For  the  degree  of  face  threat  (Wx),  we  scored  the  following: 


•  The  power  relationship  is  identical  to  the  above  P(S,H)  =  300. 

•  The  social  distance  between  them  should  also  be  identical  to  the  above. 

•  The  imposition  of  this  request  could  be  large,  especially  in  the  absence  of  the 
reminder  that  it  is  in  the  mayor’ s  interest,  but  it  remains  essentially  the  same 
request  as  in  the  previous  vignette.  Thus,  we  keep  the  “raw”  imposition  score  the 
same:  Rx  =  10. 

Thus,  the  imbalance  score  for  this  vignette  would  be:  40  -  313  =  -273.  This  again  seems 
to  be  about  right — a  score  much  less  than  zero  for  a  vignette  that  was  intended  to  include 
much  more  threat  than  redress. 

An  evaluation  similar  to  that  described  above  was  carried  out  for  a  total  of  eight  vignettes 
and  the  quantitative  algorithm  tracked  predictions  for  rude,  polite  or  nominal  perceived 
etiquette  levels  very  closely.  As  shown  in  Figure  4  all  vignettes  that  were  intended  as 
“nominal”  interactions  (that  is,  using  about  the  amount  of  redress  as  would  be  expected  in 
American  culture  for  the  amount  of  redress  offered)  scored  within  +/-  100  points  of  zero. 
All  vignettes  that  were  expected  to  be  seen  as  over-polite  scored  well  higher  than  100 
points;  while  all  that  were  expected  to  be  seen  as  overly  rude  scored  substantially  less 
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than  -100  points. 

While  the  above  example  was  based  on  one  individual’s  scoring  assessments  (Dr. 
Miller’s),  we  have  since  replicated  this  work  with  two  other  “raters”  following  a  brief 
training  session  from  Dr.  Miller.  Each  rater  then  scored  the  vignettes  according  to  the 
guidelines  as  described  above  (and  in  more  detail  in  our  training  documentation).  The 
results  of  the  three  raters’  scores  for  the  eight  etiquette  scenarios  were  then  statistically 
compared.  The  top-level  imbalance  metric  (lx)  showed  a  Robinson’ s  A  correlation  of 
.931  among  the  three  raters  across  the  8  vignettes,  and  the  two  major  subfactors  (Face 
Threat  Weight — Wx  and  composite  Redress  Value — V(AX))  the  Robinson’s  A 
correlations  were.950  and  .863  respectively.  These  values  are  all  well  above  traditional 
correlation  thresholds  of  .7  or  .8  for  multiple  judge  rating  correlations.  Thus,  this  study 
lends  weight  to  the  belief  that  we  have  identified  a  reliable  method  of  scoring  the  degree 
of  politeness  vs.  expectations  in  social  discourse — at  least  within  an  American  cultural 
setting. 

While  the  above  evaluation  used  trained  raters,  familiar  with  both  the  Brown  and 
Levinson  theory  and  with  our  own  assumptions  about  how  to  rate  redress  and  threat 
behaviors,  we  subsequently  conducted  an  experiment  wherein  22  American  college 
students  (respondents  to  flyers  and  unaware  of  our  theory  or  model),  also  rated  the  same 
8  vignettes.  Participants  reviewed  a  “backstory”  describing  the  participants  in  the 
vignette  and  then  answered  a  series  of  questions  about  their  relationships.  They  then  read 
the  specific,  verbal  interaction  (i.e.,  utterance),  and  answered  (using  Likert  scales) 
questions  about  their  perceptions  of  the  actors,  their  relationships,  the  degree  of 
politeness  used  and  whether  or  not  they  regarded  the  interaction  as  normal,  rude  or  overly 
polite. 

Correlations  between  participants’  ratings  of  the  model  parameters  and  our  own  remained 
very  high.  Comparing  the  project  team’s  mean  rating  with  the  untrained  participants’ 
mean  rating  using  Pearson’s  coefficient  showed  correlations  of  .867  for  power  ratings, 
.881  for  social  distance,  .766  for  imposition  and  .892  for  overall  imbalance/politeness 
ratings.  The  overall  politeness  correlation  was  significant  at  the  p<.01  level  (two-tailed). 

Participants  were  also  asked  whether  they  changed  beliefs  about  the  power  and  social 
distance  values  after  they  had  seen  the  utterances  used.  Since  some  vignettes  used 
utterances  expected  to  be 
nominal  while  others  used 
either  unexpectedly  high  or 
low  politeness  levels,  we 
hypothesized  that  if  our 
model  were  correct,  then 
more  participants  would  be 
willing  to  change  their 
ratings  after  seeing  the 
vignettes  with  “off- 
nominal”  politeness  rather 
than  those  with  nominal 
politeness. 

As  shown  in  Figure  5  a 
higher  percentage  of 
participants  reported 
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Rgure  5.  Percentage  of  participants  wanting  to  change  their 
estimates  of  Power  Difference  (P)  and  Social  Qstance  (D)  after 
hearing  the  actors’  utterances. 
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willingness  to  change  their  ratings  in  response  to  the  off-nominal  vignettes,  though  this 
effect  was  more  pronounced  for  Power  Difference  (P)  than  for  Social  Distance  (D) 
ratings.  A  paired-samples  t-test  on  the  mean  values  for  the  four  nominal  vs.  four  off- 
nominal  vignettes  showed  that  significantly  more  participants  wanted  to  change  their 
estimate  of  Power  Difference  after  reading/viewing  the  Off-nominal  behaviors  than  the 
Nominal  ones  (t=-4.85,  df=3,  p<.05).  A  similar  test  for  the  Social  Distance  parameter 
was  not  significant  (t=- 1 . 1 86,  df=3,  p>.2)  but  trended  in  the  same  direction. 

In  general,  these  data  support  our  interpretation  and  implementation  of  Brown  and 
Levinson’s  model  and  the  claims  that  unexpected  (i.e.,  off-nominal)  amounts  of  redress 
prompt  people  to  reinterpret  their  beliefs  about  context — specifically,  their  beliefs  about 
the  power  and  social  distance  parameters.  In  this  study,  participants  proved  more  willing 
to  review  their  perceptions  of  power  relationships  than  social  distance.  This  may  be  a 
function  of  the  marked  power  relationships  in  the  vignettes  (involving,  as  they  did, 
soldiers  and  civilians)  or  it  may  reflect  a  more  general  tendency  among  Americans  to 
seek  explanations  for  politeness  variations  in  power  dimensions  first. 

6.  Implementing  the  Etiquette  Algorithm 

We  have  been  developing  the  concept  of  computational  and  machine  representations  of  human 
face  threat,  redress,  etiquette  and  politeness  under  IR&D  since  2001.  In  this  section  we 
describe  the  successful  application  of  the  etiquette  algorithm  in  a  variety  of  military-relevant 
contexts,  along  with  some  of  the  features  which,  we  believe,  make  it  useful  for  future 
applications. 

For  example,  in  2004  we  began  our  work  with  the  Brown  and  Levinson  theory  by  creating  and 
evaluating  the  reminder  texts  for  an  in-home  medication  reminding  system  for  Honeywell. 
Subsequendy,  SIFT  and  our  partner,  Dr.  Lewis  Johnson  (then  at  USC’s  CARTE  labs),  won 
and  completed  Phase  I  of  DARPA  SBIR  SB041-009  under  Dr.  Ralph  Chatham,  as  PM.  In  this 
work  we  developed  a  “believability  metric”  for  assessing  and  predicting  when  a  given  level  of 
redressive  behavior  from  a  computerized  game  character  would  be  perceived  as 
“unbelievable”  by  a  human  player  or  observer.  Our  approach  yielded  the  face  threat  and 
redress  algorithm  described  above.  We  completed  this  work  with  a  successful  evaluation  of 
believability  predictions  over  a  set  of  scenarios  whose  degree  of  “believability”  we  had 
systematically  varied  to  have  high,  low  or  believable  amounts  of  redress  (see  Figure  4.  in 
section  5.5). 

In  work  funded  by  DARPA  under  the  Force  Multipliers  for  Urban  Operations  program  (  BAA 
04-031,  Dr.  Ralph  Chatham  again  as  PM),  we  completed  an  implementation  and 
demonstration  of  an  algorithm  based  on  Brown  and  Levinson’s  work  as  described  above.  Our 
demonstration  illustrated  the  use  of  this  algorithm  in  the  context  of  a  language  training  game  in 
which  it  enabled  characters  to  both  recognize  the  degree  of  politeness  directed  at  them  and  to 
reason  about  the  level  of  politeness  to  be  used  in  an  interaction  they  themselves  issue  in 
keeping  with  other  goals  the  character  may  have.  We  used  swappable  culture-modules: 
libraries  of  culture-specific  behavior  evaluations  enabling  serious  game  characters  to  both 
generate  and  recognize  the  threat  values  of  communicative  moves  [7].  The  results  of  this  work 
give  us  the  ability  to  represent  alternate  cultural  perspectives  by  swapping  knowledge  bases 
(including  body  behaviors)  thereby  dramatically  reducing  the  cost  and  effort  of  developing 
character  behavior. 

This  work  led  to  the  further  development  of  the  Interactive  Phrasebook,  described  above 
(SBIR  OSD06-CR5),  completed  in  2007.  The  interactive  Phrasebook  made  use  of  our  existing 
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etiquette  algorithm  and  embeded  it  in  a 
broadly  appbcable  mission  rehearsal  tool  for 
cross-cultural  interactions. 

Subsequent  work  in  this  area  (completed  in 
2009  under  SBIR  AF06-069)  took  our  social 
interaction  representations  and  applied  them 
in  a  series  of  human  experiments  to 
determine  how  variations  in  politeness 
interact  with  cultural  differences  to  predict 
compliance  attitudes  and  behaviors  (trust, 
affect,  reaction  time,  accuracy)  in  response  to 
directives:  instructions,  orders,  requests,  etc. 

[6].  This  work  provides  rich  empirical  data 
to  support  and  extend  our  interaction 
perception  models  to  adaptable  culture 
specific  models  of  behavior  and  action. 

Next,  we  will  explain  details  from  some  of 
these  projects  and  highlight  the  implications 
for  SSIM. 

6. 1  Example  of  interaction  in  Pashtun  cultural  context 

As  mentioned  above,  our  model  extends  politeness  perceptions  to  incorporate  culture-specific 
effects  on  attitudes,  perceptions  and,  ultimately,  the  behaviors  of  an  observer.  The 
computational  implementations  of  our  model  have  not  only  proven  valid  at  tracking  perceived 
politeness  in  American  cultural  contexts  [4],  but  also  adequately  representing  Pashtun  and 
Iraqi  cultural  interactions  [7].  We  will  explain  a  partial  example  (drawn  from  the  Pashtun 
culture,  as  incorporated  in  a  language  and  culture  training  game)  to  clarify  our  approach.  Note 
that  a  video  demo  of  our  EtiquetteEngineTM  for  this  scenario  is  available  at 
http://www.sift.net/demos/etiquette-quotient. 

Say  a  trainee  has  the  role  of  a  sergeant  whose  mission  is  to  enlist  the  aid  of  a  local  village 
headman  (the  “Malek”)  in  building  a  clinic.  This  interaction  might  be  viewed  as  a 
specific  instance  of  a  general  scenario  type — an  “Ask  a  Favor”  interaction.  Figure  6. 
illustrates  the  anticipated  interaction  moves  that  the  trainee  wi  1  I/m  ay  make  in  preparing  to 
ask  a  favor.  This  structure  is  not  a  strict  pathway.  Not  only  are  there  multiple  branching 
points,  but  each  block  represents  a  wide  variety  of  possible  communicative  behaviors  as 
described  below.  The  trainee’s  “moves”  could  be  constrained,  by  the  game,  to  fall  into 
the  particular  types  represented,  or  a  more  freeform  approach  might  be  implemented — 
but  these  represent  the  moves  that  the  Malek  will  expect,  and  also  those  that  the  designer 
has  deemed  important  for  training  in  this  scenario.  Which  specific  combinations  of 
phrases,  gestures,  posture,  prosody,  etc.  the  trainee  chooses  to  use  at  each  step  in  the 
scenario  will  influence  the  Malek’ s  response  and,  hence,  the  course  of  a  particular  “run” 
through  the  scenario. 

Note  that  these  “moves”  and  their  framework  as  a  pathway  for  an  “Ask  a  Favor”  scenario 
represent  a  precursor  to  the  “micro-games”  concept  we  develop  somewhat  further  in 
section  9  below. 

Consider  the  “Recognition”  step  in  this  path.  Let’s  assume  that  they  have  just  met  and 
exchanged  a  round  of  initial  greetings  and  introductions.  After  being  introduced,  the 
trainee  “recognizes”  the  introduction  by  choosing  a  communicative  act  including  the 
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Figure  6.  Trainee  pathway  for  an  "Ask  a  favor" 
scenario  or  "mcro-garre". 
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phrase  “staasee  de  lidelo  tsexa  xoshala  shwem,  saaheb”  with  a  double-handed  handshake 
gesture — meaning,  “I’m  very  happy  to  meet  you,  sir”  with  a  very  warm  and  friendly 
gesture  and  facial  expression.  In  order  to  score  the  Malek’s  interpretation  of  this 
interaction,  we  need  to  know  what  the  Malek  thinks  about  the  power  and  familiarity 
relationship  between  himself  and  the  trainee,  his  perception  about  the  imposition  of  this 
type  of  communicative  move  (a  “recognition”)  at  this  point  in  an  interaction  thread,  and 
finally,  his  understanding  of  the  various  redressive  behaviors  which  might  be  used  and 
their  relative  values. 

P,  D,  R  and  C  are  scored  on  +/-1000  point  scales  we  described  in  section  5.4  above. 

Their  application  to  this  example  was  as  follows: 

•  The  Malek  believes  he  has  slightly  more  power  than  the  trainee  does  (P  =  20) 

•  The  Malek  believes  that  he  and  the  trainee  are  not  quite  as  distant  as  complete 
strangers  would  be — they  have  mutual  acquaintances  and  a  mutual  objective  (D  = 
15) 

•  There  is  a  very  slight  imposition  to  the  trainee  taking  the  initiative  in  offering  his 
RECOGNITION  speech  act  before  the  Malek  recognizes  him,  but  this  value  isn’t 
large  in  this  culture  at  this  stage  of  the  conversation  (R=5). 

The  trainee  used  several  polite  redressive  strategies  here.  The  values  and  scores  for  these 
are: 


•  10  points  of  redressive  value  for  the  use  of  an  exaggerate  approval  strategy  (a  type 
of  positive  redress):  “I’m  very  happy  to  meet  you,  sir.” 

•  15  points  for  the  double-handed  handshake  gesture. 

•  20  points  of  redressive  value  for  the  use  of  formality,  a  form  of  giving  deference,  a 
negative  redress  strategy:  (this  greeting  is  much  more  formal  and  uses  formal 
pronouns  than  other  possible  ones) 

•  20  points  of  redress  for  an  honorific,  a  negative  redress  strategy:  “I’m  very  happy 
to  meet  you,  sir.” 

Given  these  values,  we  can  compute  the  Malek’s  perception  of  this  exchange  using  the 
equations  described  in  5.3  above. 

B0:lx  =  B0:V(AX)  -  B0:WX 
B0:lx=  (15+30+20+25) -(20+15+5+0) 

B0:lx=  100-40  =  60 

The  Imbalance  value  of  60  is  moderately  over-polite — characteristic  of  something  a  bit 
warmer  than  the  Malek  would  have  expected  from  the  Trainee,  but  not  so  extreme  as  to 
make  him  question  interpretations  and  motives.  It  should  also  be  noted  that  the  score 
would  be  lower  if  body  behaviors  would  not  be  taken  into  consideration.  In  fact,  this 
exchange  is  likely  to  put  him  into  a  good  frame  of  mind  for  future  negotiations.  Nearer 
term,  the  Malek  will  choose  his  next  utterance  in  his  scenario  pathway  (not  shown)  so  as 
to  match  the  politeness  that  the  Trainee  has  shown  him. 

Note  that  scores  such  as  those  listed  above  are  not  just  “best  judgment”  on  the  part  of  a 
designer.  We  have  reduced  our  techniques  to  a  coding  manual,  and  the  approach  has 
been  tested  for  small  data  sets  across  three  cultural  groupings  (American,  Iraqi  and 
Pashto  cultures)  with  good  success.  We  have  completed  inter-rater  reliability  studies  and 
compared  trained  raters  with  naive  ratings  from  American  students,  obtaining 


SIFT-SSIM-Report-v08-fin 


19  of  74 


12/1/2014  8:22:00  PM 


s 


Smart  Information  Flow  Technologies 


correlations  of  .950  and  .863  respectively — as  described  above  (cf.  section  5.5  and  Figure 
4). 

In  practice,  we  obtained  the  values  for  Pashto  interpretations  through  a  series  of 
interviews  and  evaluations  of  the  algorithm’s  outputs  conducted  with  a  native  Pashto 
speaker.  But  acquiring  the  knowledge  to  develop  these  knowledge  bases  is  a  process  we 
have  not  formalized  to  any  significant  degree.  We  believe  that  the  Brown  and  Levinson 
model,  with  our  extensions  (including  a  framework  to  score  nonverbal  language)  and 
formalization  of  it,  provides  an  excellent  basis  on  which  to  structure  efficient  and 
reusable  knowledge  acquisition,  and  that  role  playing  is  an  excellent  approach  to 
observing  the  use  of  verbal  and  nonverbal  politeness  behaviors  in  a  reasonably  realistic 
context,  as  the  next  example  will  show. 

6.2  A  Reusable,  Model-based  Framework  for  Cultural  Knowledge 

The  use  of  Brown  and  Levinson’s  model  in  a  module  for  reasoning  about  social 
interaction  behaviors  ensures  universal  reasoning  about  and  scoring  of  abstract  politeness 
“moves”.  The  model  already  provides  guidance  to  the  types  of  redressive  behaviors,  the 
dimensions  that  influence  face  threat  and  even  guidance  about  the  likely  relative  value  of 
different  redress  strategies.  While  scales  need  to  be  normalized  and  anchored  for  each 
new  culture  and  specific  data  represented  for  the  individuals,  interaction  moves  and 
redressive  behaviors  used  in  each  scenario,  the  model  greatly  simplifies  this  task.  First,  it 
provides  a  reusable  algorithm  which,  in  turn,  identifies  the  dimensions  which  need  to  be 
captured  (P,  D,  R  and  C)  for  each  individual  in  a  scenario  and  provides  a  taxonomy  for 
the  categories  of  redressive  actions  to  be  looked  for  in  a  culture  -  these  are  both  verbal 
(e.g.,  apologies,  honorifics,  avoiding  assumptions  of  compliance,  etc.)  as  well  as  non¬ 
verbal  (gaze,  proximity,  posture,  facial  expressions)  actions. 

Second,  such  knowledge,  once  captured,  can  be  extensively  reused.  Knowing  the 
P,D,R,C  values  for  the  Malek  and  the  sergeant  above,  I  can  compute  the  Malek’s 
perceived  threat  for  any  other  interaction  whose  redressive  moves  have  been 
characterized—  what  if  the  sergeant  failed  to  use  the  “sir”  above?  What  if  he  omitted  the 
handshake,  or  used  a  less  warm  single-handed  gesture?  What  if  he  the  sergeant  wouldn’t 
look  at  the  Malek  while  speaking?  What  if  he  used  a  more  casual  recognition  like 
“Pleased  to  meet  ya!”?  Or  what  if  the  Malek  demonstrates  his  higher  status  with  a 
dominating  pose?  Etc.  Similarly,  if  a  new  individual  is  of  interest,  we  just  need  to 
represent  the  pairwise  P,D,R,C  perceptions  for  the  new  individual  and  the  entire  scenario 
can  be  repeated.  If  we  were  interested  in  a  scenario  in  which  the  sergeant  asks  a 
shopkeeper  (lower  P  than  the  Malek)  for  help  locating  a  shop  selling  batteries  (lower 
imposition),  all  combinations  of  communicative  moves  above  can  be  reused — and  new 
computations  will  show  how  they  affect  the  shopkeeper  in  this  scenario.  Once  a 
character  or  communicative  act  is  represented  for  one  scenario,  its  scores  can  be  reused 
for  later  scenarios  where  a  designer  chooses  it  as  relevant. 

Finally,  any  such  module  will  need  to  be  equipped  with  culture-specific  knowledge  bases 
(as  illustrated  conceptually  in  Figure  7),  however,  to  enable  reasoning  from  the  observable 
behaviors  in  a  culture  (e.g.,  pursed  lips  or  a  rigid  hand-to-eyebrow  salute)  to  the  abstract 
etiquette  “moves”  (and  therefore,  politeness  implications)  over  which  the  model’s 
parameters  are  scored.  This  has  the  practical  implication  that  the  general  social 
interaction  reasoning  of  an  automated  system  can  be  effectively  modularized,  and,  thus, 
large  savings  in  simulation  code  development  can  be  realized.  For  example,  if  we  move 
from  Pashtun  to,  say,  Albanian  culture,  much  of  the  knowledge  described  above  will  have 
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to  be  recaptured — 
but  not  all,  and 
what  is  already 
implemented  will 
guide  and  speed 
what  needs  to  be 
learned.  Sergeants 
will  still  need  to 
ask  for  favors  from 
shopkeepers  and 
village  leaders; 
hence  the  core 
trajectory 

represented  in  Figure  6  can  be  reused,  perhaps  with  modifications  to  branches  and  specific 
steps.  Power  differences  and  degrees  of  imposition  may  differ  somewhat  (a  shopkeeper 
may  hold  more  or  less  status  than  in  Afghanistan),  and  of  course,  specific  redressive  acts, 
body  behaviors  and  language  phrases  will  differ.  But  while  the  redressive  value  of  the  use 
of  honorifics  may  be  the  same,  the  accompanying  gestures  and  postures  may  impact  the 
imposition  in  different  ways. 

Furthermore,  basic  game  storylines  or  training  modules  and  even  specific  characters  can 
be  easily  transposed  from  one  cultural  milieu  to  another — enabling  the  village  priest  who 
the  player  had  to  interact  with  to  get  intelligence  information  in  a  Kosovo  training  game 
to  take  on  the  culture-specific  behaviors  and  reactions  (though  not,  without  further  work 
not  reflected  here,  the  appearance)  of  an  imam  in  an  Iraqi  training  game — simply  by 
loading  a  new  module  of  cultural  knowledge.  In  each  case,  new  knowledge  bases  of 
culture-specific  politeness  behaviors  would  need  to  be  developed  (and,  of  course, 
checked  for  accuracy)  for  each  new  simulation,  but  the  core  game  storyline(s)  and 
character  roles,  general  actions,  motivations,  capabilities,  etc.,  could  remain  unchanged. 

In  our  implementation,  this  means  that  the  core  algorithm  for  computing  perceived 
politeness  can  be  used  to  emulate  the  behaviors  and  perceptions  of  any  culture  with  no 
modifications.  Culture-specific  knowledge  about  redressive  behaviors  and  their  values, 
as  well  as  about  the  factors  that  constitute  power,  social  distance  and  imposition,  can  be 
stored  in  separate  knowledge  bases  which  we  refer  to  as  “cultural  modules”.  This 
architecture  is  depicted  in  Figure  7.  By  swapping  cultural  modules  we  can,  almost  literally, 
provide  an  NPC  with  a  “brain  transplant”.  The  game  character  portraying  the  Malek  in 
Pakistan  can,  with  the  flick  of  a  software  switch,  instead  think  with  the  perceptions  and 
biases  of  a  Kosovar  village  priest.  While  the  construction  of  culture-specific  modules  of 
relevant  etiquette  knowledge  is  a  non-trivial  task  in  its  own  right,  the  framework  of  our 
model  can  serve  to  guide  and  limit  such  knowledge  acquisition  and  representation  and, 
once  implemented,  both  the  scaling  arguments  made  above  and  the  reusability  of  the  core 
reasoning  algorithm  provide  further  advantages  in  developing  believable  characters 
representing  alternate  cultural  backgrounds  and  perspectives.  Granted,  this  does  nothing 
to  alter  the  graphical  appearance  of  the  character  or  its  surroundings  (arguably  more 
resource  intensive  software  development  considerations),  it  nevertheless  represents  an 
important  innovation  which  enables  more  cost-effective  development  of  culture-specific 
emulations  in  gaming  and  training  software. 
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Figure  7.  The  role  of  "culture  modules"  of  culture-specific  politeness  knowledge 
in  our  implementation  of  the  Brown  and  Levinson  model. 
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6.3  Implementation  of  the 
Etiquette  Algorithm  into 
a  virtual  training  game: 

Tactical  Language 
Training  System  (TLTS) 

The  implementation  of  the  etiquette 
algorithm  into  the  language  training 
game  Tactical  Language  Training 
System  (developed  by  the  University 
of  Southern  California’s  CARTE 
Laboratory  [8])  demonstrated  our 
algorithm’s  ability  to  inform  both  the 
perceptions  and  the  reactions  of 
simulated  characters — and  with  less 
software  development  time  than 
traditional  scripting  approaches.  TLTS 
is  a  first-person  game/simulation  designed  to  teach  soldiers  “tactical”  versions  of  a 
language  (to  date,  Lebanese  Arabic,  Iraqi  Arabic  and  Pashto,  a  language  spoken  along  the 
Afghanistan/Pakistan  border  demonstrating  the  ability  of  our  approach  to  provide  at  least 
reasonable  knowledge  and  use  of  politeness  levels  in  a  culture  different  from  American 
English. 

The  trainee  navigates  scenarios  wherein  s/he  must  interact  with  simulated  local 
inhabitants  to  accomplish  an  overall  mission:  e.g.,  the  trainee  might  need  to  ask  a  group 
of  young  men  in  a  cafe  (Figure  8.)  who  the  leader  of  the  village  is  in  order  to  progress  to 
the  next  scenario,  wherein  s/he  will  meet  the  village  leader.  By  using  specific  phrases 
and  gestures,  the  trainee  must  convince  the  young  men  to  provide  the  information.  TLTS 
uses  speech  recognition  techniques  to  process  the  trainee’s  verbal  utterances  in  the 
selected  language,  and  offers  a  set  of  mouse-selectable  gestures  (such  as  taking  off  one’s 
hat  or  sunglasses,  covering  one’s  heart,  shaking  hand,  etc.)  to  accompany  actions.  If  the 
trainee’s  Arabic  is  not  up  to  the  task,  s/he  will  fail,  but  more  than  simple  language  skills 
are  required.  TLTS  is  also  concerned  with  politeness  in  word  and  gesture.  If  the  trainee 
is  rude  by  local  cultural  standards,  the  men  may  well  conclude  that  s/he  is  a  spy  and 
refuse  to  offer  any  information. 

Here  the  Etiquette  Engine  (EE)  operated  in  two  alternate  modes  with  TLTS,  as  illustrated 
in  Figure  9.  First,  for  a  game  character  to  recognize  the  level  of  etiquette  directed  at  it,  EE 
received  as  input  from  TLTS  the  recognized  utterance  (as  transcribed  text)  from  TLTS’s 
speech  recognition  algorithms,  along  with  any  user-selected  gestures.  EE  maintained  a 
knowledge  base  of  beliefs  that  each  individual  held  that  enabled  it  to  compute  the  relative 
power  and  social  distance  between  each  pair  of  characters.  A  second  knowledge  base 
contained  data  on  that  character’s  beliefs  about  the  relative  imposition  of  a  type  of  speech 
act  and  the  redressive  value  of  sets  of  politeness  behaviors.  These  knowledge  bases 
could  be  varied  from  individual  to  individual  with  inheritance  for  cultural  groupings.  By 
consulting  the  knowledge  bases  appropriate  to  the  character  of  interest,  we  could 
determine  that  characters’  beliefs  about  each  of  the  values  in  the  equations  above  and, 
thereby,  compute  a  perceived  imbalance  score  for  that  utterance  for  that  character. 

The  second  function  of  the  EE  was  to  recommend  “communicative  acts”  (CACTS — 
combinations  of  verbal  utterances  plus  non-verbal  content)  for  characters  to  deliver  in 
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TLTS 


Who  is  saying  what? 

To  Whom? 

How  (gesture,  with  hat  off,  etc.)? 


What  "etiquette 
imbalance"  was  used? 


TLTS 


What  would  the 
character  want  to 
do  in  response 
(content  plus 
etiquette 
imbalance)? 


accordance 
with  their 
etiquette 
goals.  The 
etiquette 
goal  could 
be 

expressed 
as  a 

modulation 
of  the  status 
quo  (i.e., 
the 

character 
might  want 
to  be  more 
or  less 
polite  or 
neutral  for 
the  current 
context), 
and 

therefore,  could  be  expressed  as  a  positive  or  negative  imbalance  number  ranging  from 
positive  to  negative  1000.  The  EE  then  examined  alternate  CACTs  of  the  type  requested 
(GREETING_RESPONSE)  and  selected  or  composed  one  whose  legal  combination  of 
redressive  acts  was  approximately  equal  to  the  desired  imbalance  number. 


What  CACT  will  achieve 
that  intention  (for  these 
participants)? 


TLTS 


Realize  it  (in 
animation  and 
speech) 


Figure  9.  Architecture  of  SIFTs  EE  integrated  in  USCs  TLTS. 


Prior  to  our  involvement,  TLTS  used  either  a  traditional  scripting  approach  or  a  complex, 
first-principles  reasoner  emulating  human  goal -based  reasoning  and  decision  making  [9]. 
Neither  of  these  was  an  entirely  satisfactory  solution.  Traditional  scripting  approaches 
greatly  limited  the  range  of  etiquette  situations  which  the  trainee  could  experience 
because  they  required  extensive  effort  to  implement  a  range  of  alternatives  and,  in  the 
absence  of  such  effort,  were  “brittle” — that  is,  they  provided  acceptable  behavior  only  for 
the  narrow  path  of  interactions  which  had  been  scripted.  When  novel,  unscripted 
behaviors  were  attempted  in  the  context  of  the  game,  character  behaviors  would  either  be 
inappropriate  or  absent  completely.  For  example,  while  a  small  range  of  greeting  and 
response  options  were  encoded,  it  was  entirely  possible  for  the  trainee  to  use  an  insult  in 
the  context  of  the  greeting  with  no  ill  effects  since  the  game  did  not  recognize  insult 
phrases.  Similarly,  it  had  limited  sensitivity  to  the  “layering”  and  combinatorics  of 
politeness  behaviors  (e.g.,  using  an  honorific  and  an  apology  and  impersonalization  vs. 
using  only  an  honorific). 

The  use  of  the  detailed  goal-based  reasoning  module,  PsychSim,  eliminated  the  problem 
of  brittleness  in  principle,  but  not  always  in  practice.  PsychSim  offers  a  rich  and  deep 
representation  of  human  goal-based  reasoning  and  includes  a  similarly  deep  reasoning 
about  the  intentions  of  other  agents.  This  complexity,  was  far  richer  and  more  general 
than  the  etiquette  and  politeness  reasoning  we  proposed  and,  thus,  even  less  brittle.  The 
problem  was  that  the  richness  PsychSim  offered  resulted  in  a  need  for  substantial 
development  effort  and  produced  comparatively  unpredictable  results.  When 
development  effort  was  not  adequate,  incorrect  interaction  behaviors  could  result.  Even 
when  it  was  adequate,  however,  the  results  were  not  always  as  controllable  as  was  desired 
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for  a  training  simulation 
which,  after  all,  needed  to 
adhere  to  a  reasonable 
“lesson  plan”  and 
progression  through  the 
stages  of  the  game. 

Our  Etiquette  Engine 
represented  something  of  a 
middle  ground  between 
these  approaches.  By 
parsing  the  perception  of 
politeness  into 
subcomponents  (power, 
social  distance,  imposition,  character  and  redressive  values),  it  opens  the  possibility  of 
recombining  elements  to  greatly  expand  the  set  of  possible  utterances  captured  in  a 
system — just  as  understanding  vocabulary  and  the  rules  of  syntax  enable  the  construction 
of  all  possible  sentences  in  a  language.  Unlike  the  linear  scalability  of  traditional 
scripting  approaches  to  representing  social  interaction  behaviors  in  games  or  simulations, 
where  each  subsequent  interaction  must  be  developed  from  scratch  with  essentially  the 
same  cost  in  labor  as  the  one  before  it,  there  is  reason  to  believe  that  our  approach  scales 
geometrically. 

In  fact,  during  our  work  with  the  TLTS  we  demonstrated  this  scalability  by  acquiring  the 
knowledge  for  and  encoding  our  first  set  of  42  “Perception  Scores”  (PSs — how  one 
observer  perceives  the  politeness  of  one  specific  communication  uttered  by  a  specific 
speaker-hearer  pair)  at  the  rate  of  2.33  PSs/hour,  but  the  next  set  were  acquired  at  19.89 
PSs/hour,  and  the  final  set  of  more  than  2000  PSs 
were  acquired  at  the  rate  of  48.96/hour.  This 
progression  is  graphed  in  Figure  10.  In  short,  it 
becomes  easy  to  recombine  previously  scored 
elements  to  generate  the  product  set  of  possible 
communicative  acts  each  of  which  will  then  be 
automatically  scored  on  the  basis  of  those 
previously  scored  elements. 


6.4  Interactive  Phrasebook 

Our  final  major  implementation  effort  of  the 
Etiquette  Engine  and  its  associated  computational 
algorithms  was  the  Interactive  Phrasebook— a 
prototype  culture  training  and  mission  rehearsal  aid 
developed  under  a  previous  Army  SBIR. 
Phrasebook  integrated  verbal  and  non-verbal 
aspects  of  communication  into  a  model-based 
architecture.  Several  candidate  modifications  are 
described  below.  Sample  interaction  screens  from 
the  prototype  we  developed  are  presented  in  Figure 
1 1  and  a  video  demonstration  is  available  at 
http://www.sift.net/demos/phrasebook.  Features  of 
our  Interactive  Phrasebook  prototype  included: 


^  *•  " 

|<|A)-mad 

Cultue:  |Axab  *| 

Character:  |0  (neutral)  H 
Occupation:  |anck  Layer 

Relationships:  - 

Actor:  |Husayn 
Character:  |2  »| 

Power  Distance:  ir=?== 
Sccial  Distance:  . 


Select  >•,  IM.  i  Defaults  Hj* 


Figure  11.  Sample  screen  from  SIFTs 
Interactive  Phrasebook-  an 
implementation  of  the  Biquette  Engine 
cteigned  to  be  used  as  a  portable 
rehearsal  aid. 


Rgure  1 0.  Scale  up  data  for  Perception  Scores  represented  during 
work  with  TLTS. 
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•  Implemented  on  a  Hewlett  Packard  iPAQ  PDA  running  the  Windows  CE  operating 
system.  Since  the  development  of  this  prototype,  mobile  technology  has  advanced 
considerably.  However,  the  suggested  architecture  can  be  relatively  easy  integrated  in 
a  mobile  application  for  several  current  operating  systems. 

•  Scenarios  selected  from  library  by  user  based  on  current  needs. 

•  Scenario  characters  and  roles  selected  and  tuned  from  stock  characters — variations  in 
P,  D,  R  and  C  were  under  the  user’s  control  and  thus  the  basic  scenario  could  be 
“tuned”  (e.g.,  asking  a  big  favor  from  an  important  person  vs.  small  favor/familiar 
person,  etc.) 

•  Interaction  moves  (including  gesture,  posture,  prosody,  etc.)  were  selected  from 
menu  alternatives  in  text  (a  la  a  theater  script) — though  English,  Arabic  phonetic 
transcription  and  Arabic  script  versions  were  available.  Available  utterances  were 
synchronized  to  the  alternate  steps  in  the  scenario  (cf. Figure  6). 

•  Guidance  for  pronunciation,  gesture/posture/prosody  execution,  etc.  was  available 
through  video,  audio,  animation  and  photo  files  linked  to  potential  utterances. 

•  Character  actions  can  be  controlled  by  initial  attitude  and  attitude  adjustments  arising 
from  trainee  interaction  selections — hence  there  was  a  bias  toward  using  adequate, 
but  not  excessive,  politeness.  Our  recent  work  [7]  has  provided  us  with  multi-cultural 
data  on  the  impacts  of  face  threat  and  redress  on  trust,  affect,  compliance  rates  and 
reaction  times,  etc.  These  data  will  provide  grounding  for  more  complex  character 
reasoning  and  reaction  models.  We  are  aware  that  obtaining  compliance  may  be 
more  about  asserting  power — perhaps  through  “rudeness” — and  will  be  able  to 
develop  and  convey  such  models. 

•  Ongoing  feedback  on  perceived  threat  and  redress  can  be  provided  from  our  scoring 
algorithm,  both  before  each  utterance/body  movement  selection  and  on  an  ongoing 
basis  throughout  the  scenario  to  reflect  changing  attitudes. 

•  Reviews  of  our  Phrasebook  prototype  with  deploying  soldiers  at  Ft.  Polk  indicated 
high  regard  for  our  approach.  Quotes  included  “I  think  everyone  should  have  one” 
(from  a  specialist  training  for  his  fourth  tour  in  Iraq  and  Afghanistan)  and  “This 
could  save  lives”  (from  a  redeploying  sergeant). 

6.5  Using  Etiquette  and  Politeness  “in  reverse” 

Our  most  recent  work  has  moved  away  from  using  the  etiquette  model  to  control  the 
behaviors  or  perceptions  of  characters  in  games  or  simulations  and,  instead,  is  focusing 
on  using  it  to  guide  the  interpretations  of  politeness  behaviors  used  in  human-human 
interactions.  In  work  sponsored  initially  by  the  U.S.  Navy  [10]  and  subsequently  and 
much  more  extensively  by  NASA  [11],  we  have  been  using  the  etiquette  algorithm  and 
Brown  and  Levinson’s  politeness  model  as  a  guide  to  interpreting  human  interaction 
behavior  for  cues  to  the  relationships  that  exist  among  the  participants.  In  the  Navy 
work,  for  example,  we  were  able  to  correctly  predict  100%  of  the  core  power 
relationships  in  a  set  of  operational  chat  messages  that  took  place  over  the  course  of  a 
week  during  a  Marine  simulation  exercise.  While  this  work  extends  and  makes  continued 
use  of  the  politeness  scoring  representations,  it  does  not  (yet)  explicitly  consider  body 
behaviors  and  is  not  used  in  guiding  simulations  or  game  characters.  Therefore,  it  will 
not  be  discussed  further. 
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Instead,  we  will  turn  to  a  brief  examination  of  the  available  literature  on  a  core  set  of 
body  behaviors  for  guidance  as  to  how  to  incorporate  these  into  the  Etiquette  Engine 
representation  and  reasoning  approach. 

7.  Cross  Cultural  Body  Behavior  Data  and  Models  from  the 
Literature 

As  outlined  above,  politeness  theory  provides  several  linguistic  strategies  for  redress  (cf. 
Figure  4  and  especially  Figure  2),  however  nonlinguistic  aspects  of  politeness  may  be 
equally  important  [2],  While  Brown  and  Levinson  focus  almost  entirely  on  linguistic 
manifestations  of  politeness,  they  acknowledge  that  nonverbal  behaviors  also  play  an 
important  role  in  face  threat.  For  example,  Brown  and  Levinson  refer  to  “hesitant 
prosodies  and  kinesics”  (p.  188)  and  “hedging  through  intonation” (p.  147)  as  examples  of 
manifestations  of  negative  politeness  strategies.  While  a  number  of  politeness  strategies 
can  be  communicated  nonverbally,  the  connection  between  the  strategic  use  of  verbal  and 
nonverbal  behaviors  has  been  mostly  ignored. 

However,  there  are  several  reasons  why  nonverbal  behavior  is  at  least  as  important  (if  not 
more  important)  than  verbal  expressions. 

1)  Much  of  our  communication  is  expressed  through  nonverbal  messages. 

2)  Nonverbal  and  verbal  cues  function  cooperatively  in  a  complex  manner.  For 
example,  interpretations  of  polite  utterances  to  save  face  depend  not  only  on  the 
semantic  content,  but  also  on  the  accompanying  nonverbal  cues.  Delivering  bad 
news  (i.e.  the  statement  “we  lost.”)  with  a  smile  or  without  a  smile  leads  to 
different  interpretations,  even  though  the  linguistic  content  is  the  same.  Saying, 
“we  lost”  with  a  smile  appears  more  optimistic  and  gives  a  sense  that  this  is  “part 
of  life”,  while  saying  “we  lost”  without  a  smile  carries  more  negative  weight  and 
disappointment. 

3)  Brown  and  Levinson’s  politeness  theory  originates  in  Goffman’s 
conceptualization  of  “facework”  [2]  -  communicative  strategies  used  to  preserve 
self-esteem  and  “face”.  Goffman  suggest  that  a  lot  of  facework  occurs  through 
expressive  behavior  and  the  nonverbal  channel  of  communication.  Interpretations 
of  utterances  that  save  face  depend  not  only  on  the  semantic  content,  but  also  on 
the  accompanying  nonverbal  cues  that  are  able  to  contextualize  an  utterance. 

4)  There  is  some  evidence  that  nonverbal  communication  is  more  universal  across 
cultures  than  linguistic  communication.  First,  behavioral  research  provides  strong 
evidence  that  facial  expressions  representing  basic  emotions  are  recognized 
universally  [12].  Second,  verbal  information  is  embedded  in  different  languages 
and  words  that  may  carry  culture-specific  meaning,  which  may  be  difficult  to 
translate.  However,  there  is  also  evidence  that  neural  activity  related  to  social 
interactions  is  impacted  by  differences  in  cultures.  For  example,  East  Asian 
cultures  show  increased  neural  activity  in  the  brain  regions  related  to  inference  of 
others'  mind  and  emotion  regulation,  while  Western  cultures  show  enhanced 
neural  activity  in  the  brain  areas  related  to  self-relevance  and  emotional 
responses  during  social  cognitive/affective  processes  [13].  In  sum,  nonverbal 
behaviors  are  influenced  by  social  and  cultural  contexts,  influence  perception  of 
politeness  and  need  to  be  understood  within  its  linguistic  context  in  order  to 
interpret  meaning  accurately. 

The  influence  of  culture  on  nonverbal  expressions  of  politeness  is  relatively  unexplored, 
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however  some  studies  have  investigated  the  effects  of  nonverbal  behavior  on  perceptions 
of  social  relations.  For  example,  vocal  pitch  and  facial  expressions  have  been  found  to 
vary  with  intent  to  convey  politeness  [14].  Nonverbal  cues  of  politeness  have  been  found 
to  vary  with  status,  depending  on  culture.  Research  suggests  that  when  politeness 
strategies  are  expressed  through  different  channels  of  communication  (silent  video, 
speech,  video  and  audio)  nonlinguistic  strategy  usage  is  related  to  social  and  contextual 
factors  [15].  For  example,  cultural  context  is  a  mediating  factor  when  choosing  the 
appropriate  politeness  strategies,  i.e.  in  low-context  cultures  (American)  the  content  of 
the  communication  is  more  important  and  in  high-context  cultures  (Korea)  the  relational 
(hierarchical)  context  is  more  important  when  conveying  a  message.  Other  studies  using 
nonverbal  cues  to  manipulate  politeness  (polite/impolite)  have  shown  that  these  cues 
affect  the  perceptions  of  the  politeness  of  the  verbal  messages.  When  conveying 
criticisms,  polite  nonverbal  tone  increased  perceived  politeness  and  impolite  nonverbal 
tone  decreased  perceived  politeness  [16].  Our  own  prior  work  has  generally  tried  to 
minimize  non-  or  extra-verbal  cues  so  that  we  could  study  the  effects  of  specific 
relational  variables,  but  even  in  this  work  we  have  been  able  to  show  that  contextual 
variables  such  as  team  relationships,  power  relationships  and  gender  can  alter  perceptions 
of  politeness  as  well  as  compliance  behaviors  with  requests  accompanied  by  those 
behaviors  [17], 

However  a  systematic  organization  of  cross-cultural  body  behaviors  relating  to  politeness 
is  by  and  large  missing.  One  attempt  to  organize  displays  of  nonverbal  behaviors  in 
human  relations  is  the  use  of  a  “verticality”  dimension  [18].  The  vertical  dimension 
relates  to  power,  dominance,  status,  hierarchy,  and  related  concepts,  and  stands  in 
contrast  to  the  ‘'horizontal”  (affective  or  socio-emotional)  dimension,  which  describes  the 
emotional  closeness  of  interpersonal  relations  and  the  valence  of  feelings  and  behavior 
[19],  These,  obviously,  are  echoed  by  Brown  and  Levinson’s  P  and  D  dimensions. 
However,  distinctions  between  dominance  power  and  status  need  to  be  taken  into 
consideration.  Briefly,  dominance  includes  the  motive  to  control  others  or  the  self¬ 
perception  of  oneself  as  controlling  others,  while  status  is  a  more  ascribed  or  achieved 
quality  implying  respect  and  privilege,  but  not  necessarily  the  ability  to  control  others. 
This  stands  in  contrast  to  power,  which  is  broadly  the  capacity  or  right  to  control  others 
and  which  may  have  different  function  basis  (reward  power,  expert  power,  coercive 
power,  etc.)  [20].  Verticality  can  be  seen  as  an  overall  dimension,  including  different 
conceptual  definitions  relating  to  the  same  concept,  but  it  may  not  be  concordant  with  a 
person  across  time,  relationship  or  situation  (i.e.  a  leader  at  work  may  be  meek  at  home). 
The  dominant-subordinate  dimension  maps  also  onto  approach  and  avoidance  motives, 
with  associated  behavioral  tendencies.  Interpersonal  sensitivity  to  context  should  be  high 
for  subordinates  because  of  their  motive  to  reduce  threat  by  gathering  information  [21]. 

Because  nonverbal  behavior  is  often  tacit  and  “off-the-record”  it  is  used  to  express  and 
maintain  verticality  without  the  need  to  invoke  it  explicitly.  Henley  stated  whether 
behavior  is  associated  with  power,  affecting  or  resulting  from  it,  as  symbols  or 
expressions;  as  describing,  establishing  or  maintaining  power,  power  and  nonverbal 
behavior  are  intimately  linked  [22].  For  example,  a  recent  review  by  Hall  et  al.  [20]  links 
higher  verticality  with  less  smiling,  more  gazing,  more  lowered  brows,  more  expressive 
face,  more  nodding  less  self  touching,  more  touching  others,  more  hand/arm  gestures, 
more  bodily  openness,  less  bodily  relaxation,  more  bodily  shifting,  smaller  interpersonal 
distances,  more  vocal  variability,  louder  voice,  more  interruptions,  shorter  speech 
latencies/  less  pausing,  fewer  filled  pauses,  more  laughter,  fewer  speech  errors,  faster 
speech  (especially  for  U.S.  perceivers),  lower  voice,  and  more  relaxed  voice  (especially 
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for  U.S.  perceivers).  The  authors  also  found  that  beliefs  about  the  link  between  verticality 
and  body  behaviors  are  stronger  than  the  corresponding  actual  relations.  One  possible 
reason  for  why  this  is  the  case  is  that  stereotypes  may  be  activated. 

Nonverbal  expressions  of  verticality  are  also  often  related  to  different  behaviors 
displayed  by  women  and  men.  On  average,  women  show  more  body  behaviors  associated 
with  low  verticality  -  they  smile  more  than  men  [23],  are  more  expressive,  touch  others 
more  and  are  generally  more  sensitive  to  nonverbal  cues  [24].  However,  it  cannot  be 
concluded  with  certainty  that  associations  between  body  behaviors  and  verticality  mirror 
gender  differences  in  nonverbal  behavior. 

Other  models  that  organize  body  behavior  have  been  put  forth.  For  example,  the 
interpersonal  circumplex  theory  [25]  organizes  interpersonal  behavior  along  two 
dimensions:  1)  the  affiliation  dimension  (anchored  in  agreeableness)  and  the  control 
dimension  (anchored  in  dominance  and  submission).  It  is  a  model  for  conceptualizing 
and  organizing  interpersonal  behavior  and  motives  [26]  and  can  be  visualized  with  a 
vertical  axis  (status,  dominance,  control)  and  a  horizontal  axis  (friendliness,  warmth, 
affiliation).  The  vertical  and  horizontal  axes  have  been  associated  with  the  constructs  of 
agency  and  communion,  where  each  point  in  the  circumplex  circle  can  be  specified  by  a 
weighted  combination  of  communion  and  agency  [27].  Figure  12A  shows  a  simplified 
model  with  submission/dominance  and  ignore/love  at  opposite  ends  along  the  dimensions 
of  communion  and  agency.  Figure  12B  shows  the  same  model  where  several  attitudes  are 
mapped  onto  this  circle,  with  circles  ranging  from  the  center  to  outer  circle  indicating 
intensity.  Research  in  the  area  of  interpersonal  circumplex  theories  provides  some 
predictions  about  the  dominance-submission  dimension  of  human  behavior  in  social 
interactions.  It  predicts  that  people’s  behaviors  will  be  similar  to  interactional  partners 
along  the  affiliation  dimension  and  opposite  along  the  control  dimension,  and  that  when 


Figure  12  A  Interpersonal  circumplex  by  Isbister,  B.  Interpersonal  circumplex  by  Leary 

this  complementary  response  occurs,  the  partners  will  like  each  other  more  and  will  be 
more  comfortable  [28]. 

Cultures  have  been  shown  to  differ  with  regard  to  the  value  they  assign  to  the  vertical 
dimension  of  social  relations  [29;  30].  Thus,  cultures  can  be  regarded  as  a  source  of 
diversity  in  how  they  value  power  and  who  they  accord  it  to.  However,  data  on  cultural 
differences  suggests  that  nonverbal  expressions  of  dominance  are  recognized  and 
processed  as  transcultural  universals,  even  though  evaluations  and  interpretations  of 
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status  and  power  follow  culture-specific  values  and  attitudes.  Thus,  research  suggests  a 
division  of  power  perception  into  two  independent  layers  -  one  that  is  more  basic  and 
culturally  universal  (Dominance:  dominant-submissive,  weak-strong,  respectful- 
disrespectful  and  confident-  unconfident)  and  the  other,  which  is  more  culture-specific 
(Evaluation:  unfriendly-friendly,  believable-unbelievable,  likable-dislikeable  and  cold- 
warm-hearted)  [31].  Thus,  culture  may  determine  the  value  attached  to  a  status,  but 
perception  of  dominance  cues  (i.e.  status  cues  like  expanding  body  posture)  seem  to  rely 
on  universals,  probably  grounded  in  evolutionary  theory.  For  example,  in  cross-cultural 
business  negotiations,  cultural  display  is  able  to  reveal  different  expression  of  status  in 
different  groups,  while  observers  agree  upon  who  is  dominant  and  who  is  submissive 
[32].  Other  studies  have  found  similar  results.  Cultural  universalities  have  been  reported 
in  perception  of  nonverbal  dominance  behaviors  from  facial  expressions  [33], 
observation  of  dominant  and  submissive  postures  [34]  and  from  nonverbal  status  cues, 
even  if  those  where  not  part  of  the  own  cultural  repertoire  [32], 

In  sum,  several  models  of  nonverbal  behaviors  have  been  proposed  following  roughly  a 
vertical  and  horizontal  division,  with  stronger  cultural  impact  along  the  horizontal 
dimension.  We  will  return  to  these  frameworks  in  the  next  section  and  propose  a  way  to 
integrate  these  with  Brown  and  Levinson’s  politeness  strategies  as  well  as  specific 
nonverbal  behaviors  that  map  onto  the  outlined  theoretical  model  under  section  4. 

But  first,  we  will  report  the  results  of  a  detailed  examination  of  the  literature  on  three 
specific  types  of  body  behaviors  we  conducted  during  this  project:  gaze,  posture  and 
smile.  We  chose  these  behaviors  in  part  because  there  was  a  rich  literature  on  them,  but 
also  to  focus  our  thoughts  on  how  to  represent  information  suitable  for  integration  into 
our  Etiquette  Algorithm. 

7. 1  Gaze  -  Visual  Dominance  Ratio 

Gaze  can  transmit  essential  social  information,  i.e.  about  the  relationship  of  two 
individuals  [35]  or  turn-taking  patterns  in  conversations  [36].  It  is  a  powerful  modulator 
of  explicit  and  implicit  social  interaction  [37],  signals  potential  sources  of  reward  or 
danger  and  activates  approach-avoidance  responses  [38].  Gaze  is  also  an  important 
nonverbal  marker  of  status.  For  example,  studies  of  group  interactions  found  differences 
in  visual  attention  expressed  through  gaze  between  high-  and  low-status  group  members 
[39],  The  higher  the  status  in  the  group  the  more  likely  it  is  that  people  look  and  follow 
the  gaze  of  the  high  status  group  member.  Thus,  gaze  is  a  valuable  indicator  of  a  person’s 
status  in  a  group.  Specifically,  status  that  is  based  on  either  competence  or  dominance 
attracts  more  gaze  followers  [40],  However,  in  social  hierarchical  relationships, 
differences  between  in-  and  out-group  members  have  been  noted.  For  example,  low- 
social  status  male  rhesus  macaques  reflexively  follow  the  gaze  of  any  familiar  rhesus 
macaques,  but  high-status  macaques  selectively  follow  the  gaze  of  other  high-status 
monkeys  [41].  Similar  findings  have  been  reported  among  humans.  For  example,  during 
public  speeches,  the  gaze  of  politicians  has  been  shown  to  reflexively  capture  the  gaze  of 
in-group  voters  [42].  Thus,  gaze  can  also  be  an  informative  marker  of  in-group 
membership. 

A  highly  effective  method  to  identify  status  differences  is  the  visual  dominance  ratio 
(VDR).  The  VDR  is  defined  as  the  proportion  that  a  person  looks  at  another  while 
speaking  vs.  while  listening — as  will  be  explained  in  more  detail  below  (see  ).  The  VDR 
is  used  to  understand  power  relationships  in  face-to-face  interactions  of  two  persons  [35], 
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and  more  recently  also  in  the  context  of  groups  [43]. 

VDR  for  person  A  interacting  in  a  dyad  with  person  B  is  computed  as  shown  in  Table  3. 
In  Formula  1  (based  on  Koch  et  al.,  [43]),  the  numerator  is  the  ratio  of  A  looking  at  B  as 
a  proportion  of  the  total  speaking  time  of  A.  The  denominator  is  the  ratio  of  A  looking  at 
B  in  relation  to  the  total  speaking  time  of  B.  If  A  looks  at  B  for  the  same  proportion  in 
both  speaking  modes,  the  VDR  has  a  value  of  1 .  If  A  looks  at  B  longer  while  A  himself  is 
talking  more  than  listening,  the  numerator  becomes  larger  than  the  denominator,  and  the 
VDR  is  greater  than  1 .  In  the  reverse  case,  if  A  looks  at  B  longer  when  B  talks  than  when 
A  talks,  the  denominator  becomes  larger  than  the  numerator,  and  the  VDR  is  less  than  1 . 


Table  3.  Visual  dominance  measure  for  Dyads  and  Groups 


Higher  VDR  ratios  are  associated  with  higher  ‘verticality’  [20]  for  both  men  and  women 

and  reflective  of  a  person’s 
influence  in  groups  of  two 
people.  Persons  with 
relatively  little  power  or  status 
look  longer  at  others  while 
listening  than  while  talking 
themselves,  while  more 
powerful  persons  look  about 
the  same  amount  of  time 
while  listening  and  talking.  In 
sum,  less  powerful  persons 
look  longer  at  more  powerful 
persons,  especially  in  the  role 
of  the  listener. 


Formula  1 :  VDR  for 
dyads 


Formula  2:  VDR  for 
groups 


AlwtB  x  inn 
A  talk  total  X  luu 

AlwlB  x  |  nn 
A  listen  total  A  1UU 


Alwttotal— Alwtaway  v  i  nn 
A  talk  total  X  1UU 

A  lwl  total  -  A I  wl  away  <  nn 
talk  total  group  without  A 


Iwt:  looks  while  talks,  lwl:  looks  while  listens,  away:  looks 
away,  i.e.,  neither  at  the  entire  group  nor  at  a  certain  target 
person) 


Note  that  there  are  echoes  of 

Brown  and  Levinson’s  positive  and  negative  politeness  strategies  in  these  behaviors, 
though  perhaps  not  a  clear  distinction  between  them.  Looking  at  another  (especially 
when  s/he  is  speaking)  is  a  form  of  attending  (literally,  in  this  case)  to  the  other’s  face 
and  a  subordination  of  one’s  own  attentional  interests.  Thus,  it  is  a  form  of  deference  and 
should  behave  as  a  redressive  behavior.  Looking  away  denies  this  form  of  redress  and, 
therefore,  asserts  power  or  lack  of  social  connection. 


Several  research  studies  have  identified  the  link  between  VDR  and  perceived  expertise  on 
a  topic  [44],  objectively  measured  rank  [45],  ascribed  status  [46],  reward  power  [47]  and 
personality  dominance  [46].  For  example,  when  speaking  to  each  other,  ROTC  officers 
exhibited  ratios  of  1.06,  while  ROTC  cadets  speaking  to  officers  had  ratios  of  0.61; 
undergraduates  in  an  introductory  psychology  course  scored  0.92  when  talking  to  a 
person  they  believed  to  be  a  high  school  senior  who  did  not  plan  to  go  to  college,  but  0.59 
when  talking  to  a  person  they  believed  to  be  a  college  chemistry  honor  student  accepted 
into  a  prestigious  medical  school;  expert  men  speaking  to  women  about  a  subject  in  their 
own  field  scored  0.98,  while  men  talking  to  expert  women  about  the  women’s  field,  0.61; 
expert  women  speaking  to  non-expert  men  scored  1 .04,  non-expert  women  speaking  to 
expert  men  scored  0.54  [48]. 


However,  a  few  considerations  need  to  be  mentioned:  1)  this  behavior  occurs  primarily  at 
the  unconscious  level  [49;50],  2)  Gaze  behaviors  in  group-contexts  help  to  understand  the 
relative  position  and  influence  of  single  individuals.  3)  Sex  and  situational  factors  relate 
to  visual  power  displays.  On  average,  men  display  more  equivalent  levels  of  looking 
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while  speaking  and  looking  while  listening,  resulting  in  high  visual  dominance  ratios, 
associated  with  higher  social  power.  Women  look  more  while  listening  than  while 
speaking,  a  pattern  associated  with  lower  power  positions  [51].  4)  Cultural  differences 
need  to  be  taken  into  considerations.  While  some  studies  found  cross-cultural  similarities 
of  direct  gaze  [52],  there  is  also  evidence  that  cultures  differ  in  gaze  patterns.  For 
example,  research  found  greater  levels  of  gaze  (with  regards  to  appropriateness,  i.e. 
directionality,  frequency  or  intensity)  in  ‘contact’  cultures  (Arabs,  Latin  America, 
Southern  Europe)  compared  to  “non-contact”  cultures1  (Asians,  Indians-Pakistanis, 
Northern  Europeans)  [53,  54],  In  some  cultures  too  little  gaze  is  interpreted  as  a  sign  of 
dishonestly  and  insincerity,  while  in  other  cultures  turning  the  eyes  downward  is  viewed 
as  a  gesture  of  respect.  In  yet  other  cultures,  people  use  a  veil  to  protect  themselves  from 
the  “evil  eye”.  However,  since  the  VDR  ratio  can  be  understood  as  a  measure  of 
dominance  and  since  verticality/dominance  is  more  likely  recognized  across  cultures,  we 
conclude  that  the  VDR  can  be  indicative  for  power  relationship  interpretations 
independent  of  cultural  context.  We  will  provide  an  example  how  the  VDR  can  be 
calculated  an  applied  to  a  relevant  SSIM  sample  below  in  section  9.4. 

7.2  Posture  -  Complementarity  and  Mimicry 

Posture  is  another  important  form  of  nonverbal  communication  indicative  of  dominant- 
submissive  perceptions  of  social  relations  [55].  Subtle  changes  in  posture  can  lead  to 
different  perceptions  of  dominance.  When  people  expand  themselves  and  take  up  a  lot  of 
space,  i.e.  expanding  limbs,  they  are  perceived  as  dominant,  whereas  when  they  constrict 
themselves  and  take  up  little  space,  they  are  perceived  as  submissive  [56,  57], 

Note  that  there  are  echoes  here  too  of  the  Brown  and  Levinson  politeness  behaviors. 
Restricting  oneself  (in  this  case,  literally),  can  be  seen  as  a  means  of  reducing  or 
sacrificing  one’s  face  to  the  other’s  needs  or  desires.  Apologies,  honorifics,  indirect 
questioning  and  hedging,  etc.  are  all  negative  politeness  strategies  identified  by  Brown 
and  Levinson  (cf.  Figure 2)  that  are  a  form  of  sacrificing  or  reducing  one’s  face  in  order  to 
accord  more  prominence  or  regard  to  the  face  of  the  other  and  thus,  at  least 
metaphorically,  they  operate  similarly  to  these  physical  behaviors. 

More  erect  (less  relaxed)  posture  connotes  social  potency,  but  it  should  be  noted  that  the 
context  determines  whether  erect  posture  might  be  seen  as  nervous  and  polite  (and 
therefore  reflecting  low  verticality)  or  as  reflecting  proud,  confident  bearing  (and 
therefore  suggestive  of  high  verticality)  [20].  These  so-called  “power  moves”  [28] 
communicate  the  actor’s  likely  status  position  to  observers,  because  postural  expansion 
occurs  more  frequently  among  people  who  have  high  status  and  constriction  more 
frequently  among  people  who  are  low  in  status  [57].  However,  body  expansion  and 
constriction  not  only  influences  the  perception  of  status,  it  also  influences  the  behavioral 
responses  of  others  [20].  Posture  and  postural  responses  can  help  understand  D(S,H)  and 
P(H,S)  and  can  therefore  influence  level  of  politeness.  Changes  in  postural  responses  to 
displayed  nonverbal  behavior  -  too  -  often  happen  automatically,  reflex-like  and  without 
intention  [58],  or  even  as  preconscious  and  goal-independent  behavior. 

There  are  at  least  two  forms  of  systematic  effects  of  posture  displays  as  a  response  to  the 


A  contact  culture  is  defined  as  a  culture  in  which  people  tend  to  stand  close  together  and  touch  frequently 
when  they  interact  together.  A  noncontact  culture  is  a  culture  where  groups  tend  to  maintain  more  space  and 
often  less  touch  than  contact  cultures. 
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behaviors  of  others  [28]: 

1)  Postural  mimicry.  H  is  a  dominant  person  and  S  is  a  submissive  person.  Let’s  say 
H  imitates  the  behaviors  which  S  performs.  H  displays  dominance  behaviors  as  a 
reaction  to  S’s  display  of  dominance,  and  H  responds  to  submissive  behaviors 
with  mutual  submission.  Since  mimicry  is  effective  at  increasing  affiliation,  it  is 
most  likely  displayed  in  contexts  where  affiliation  goals  are  primary.  Mimicry 
also  depends  on  the  status  of  the  actors.  People  tend  to  mimic  high  status  people, 
a  pattern  referred  to  as  “accommodation”  [59].  Mimicry  is  heightened  when 
people  perceive  themselves  as  similar  [60],  have  aligned  goals  [61],  share 
attitudes,  want  to  empathize  [62]  or  want  the  actor  to  have  positive  perceptions 
and  like  them  [63].  It  is  related  to  factors  such  as  involvement  and  interest  [64] 
and  people  who  want  to  be  liked  and  increase  affiliation  tend  to  show  postural 
mimicry.  When  observers  see  postural  mimicry,  they  are  able  to  not  only  identify 
the  social  distance  (affiliation)  between  two  persons  but  also  deduce  whether  the 
nature  of  the  social  relationship  is  competitive  or  cooperative  and  whether  S  and 
H  belong  to  the  same  in-group  [28].  In  sum,  the  goal  of  postural  mimicry  is  to 
increase  liking,  but  it  is  highly  context-dependent.  For  example,  postural  mimicry 
is  perceived  as  less  comfortable  in  hierarchical  relationships. 

2)  Postural  complementarity.  H  responds  with  contrasting  behaviors  to  S.  Postural 
complementary  happens  when  a  dominant  display  invites  a  submissive  response 
and  vice  versa.  Some  researchers  argue  that  human  postural  expansion  and 
constriction  is  reminiscent  of  the  dominance  displays  in  other  species.  For 
example,  among  primates,  such  as  chimpanzees,  dominant  group  members  show 
body  expansion  in  order  to  appear  as  large  as  possible.  Displays  of  such  postures 
are  typically  met  with  submissive  displays  [65] .  Among  chimpanzee  colonies 
postural  complementary  leads  to  more  peaceful  relations,  while  mimicry  of 
dominant  displays  can  mark  the  beginning  of  violent  conflict.  While  evolutionary 
theories  argue  that  similar  patters  characterize  human  relations,  there  is 
conflicting  evidence  that  this  is  in  fact  the  case,  since  humans  have  not  only 
evolved  to  more  egalitarian  relationships,  but  they  are  more  conscious  and 
strategic  about  their  behaviors  in  relation  to  others,  making  complementarity  less 
common.  However,  in  hierarchical  relationships,  exhibiting  complementary 
postures  across  hierarchical  levels  leads  to  more  liking  and  less  conflict..  In  fact, 
nonverbal  complementary  and  the  comfort  and  liking  associated  with  it  may 
encourage  and  maintain  hierarchical  relationships.  In  sum,  in  hierarchical 
configurations,  postural  complementary  is  considered  more  comfortable,  even 
though  we  are  largely  unaware  of  the  functional  aspect  of  this  behavior. 

While  it  is  hard  to  predict  whether  two  individuals  will  display  postural  mimicry  or 
complementary,  we  believe  that  some  theories  can  either  predict  and/or  recommend 
adaptive  behaviors  for  any  given  situation.  For  example,  interpersonal  circumplex  theory 
(see  Figure  12)  predicts  that  people’s  behaviors  will  be  similar  along  the  affiliation 
dimension  and  complementary  along  the  control  dimension.  Other  studies  confirmed  this. 
When  S  displays  an  expanded  (dominant)  or  constricted  (submissive)  posture  H  tends  to 
complement  rather  than  mimic  those  postures.  But  when  S  and  H  are  relationship 
partners,  they  tend  to  show  mimicry  [28].  The  relationship  between  social  distance  and 
posture  is  also  summarized  in  Figure  14  in  section  7.4. 
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7.3  Smile 

Smile  is  another  form  of  nonverbal  behavior  for  the  cross-cultural  understanding  of  social 
relationships  which  may  besuitable  to  be  considered  for  integration  in  the  etiquette 
algorithm.  Among  nonhuman  primates,  all  studied  species  use  facial  gestures  to  signal 
social  status  in  order  to  help  regulate  relationships  among  conspecifics  [33].  Among 
humans  smiling  has  been  associated  with  greeting  in  both  Western  and  non-Western 
cultures  [57]  and  is  considered  a  universal  expression  of  happiness  [12].  However,  there 
is  also  evidence  for  taking  a  cultural  relativistic  approach  towards  the  universality  of 
smiling  as  an  expression  of  happiness.  One  viewpoint  states  that  expressive  displays  of 
emotional  feelings  are  basically  comparable  across  cultures.  Another  point  of  view 
divorces  the  display  of  emotions  from  their  biological  basis  and  concedes  that  display  of 
emotions  is  influenced  by  social  norms,  leading  to  large  variations  in  smiling  behaviors 
across  cultures  [67],  However,  it  is  very  likely  that  there  are  universal  similarities  in  the 
experience  and  expression  of  emotions  across  cultures,  while  there  are  still  differences  in 
the  appraisal  thereof  [33].  For  example,  Matsumoto  [68]  found  cultural  differences  for 
display  rules  for  smiling  between  US  Americans  and  Japanese.  While  his  approach  is 
based  on  Hofstede’s  individualism-collectivism  and  power  distance  dimension,  he  found 
that  American  participants  consider  the  display  of  happiness  in  public  as  more 
appropriate  than  Japanese  do.  Other  researchers  compared  the  perception  of  nonverbal 
immediacy  behaviors,  such  as  smiling,  across  participants  from  Australia,  Finland,  Puerto 
Rico  and  the  US  and  found  cultural  differenced  for  the  appreciation  of  touch.  However, 
for  smiling  or  having  a  relaxed  body  positions  no  difference  emerged  [69].  Yet  another 
study  looked  at  smiling  in  relation  to  the  social  distance  between  S  and  H.  When 
comparing  smiling  at  a  professor  versus  a  fellow  student  in  different  cultural  contexts,  all 
students  rated  themselves  as  more  likely  to  smile  towards  the  professor  than  towards  a 
fellow  student.  But,  when  asked  what  intensity  seems  appropriate,  Asian  Canadians 
preferred  an  intensity  of  37.8%  vs.  European  Canadians,  who  preferred  smiles  of  87.7% 
intensity.  Thus,  while  there  is  evidence  for  the  recognition  of  smile  as  an  expression  of  a 
generally  positive  feeling,  there  are  a  few  cultural  differences  in  the  display  rules  for 
smiling,  the  frequency  of  smiling  or  the  perception  of  smiling  individuals.  Overall,  all 
members  of  the  groups  most  often  studied  -  Asians  and  North  Americans  -  tend  to  smile 
frequently  and  their  smiles  lead  to  a  more  positive  evaluation.  But  there  is  evidence  that 
members  of  Asian  cultures  smile  somewhat  less  and  more  importantly  smile  less 
intensely  and  this  difference  may  lead  to  miscommunications  of  intent  in  a  variety  of 
social  situations. 

There  are  also  universal  gender  differences  with  regards  to  the  frequency  of  smiles,  with 
women  smiling  more  than  men.  This  tendency  is  often  associated  with  status,  where 
individuals  high  in  verticality  (often  men)  smile  less  than  women  [70;  71].  Interestingly, 
higher  levels  of  testosterone  in  men  and  women  have  been  associated  with  more  self¬ 
esteem  and  (in  men)  dominant  behavior  and  less  smiling  [72],  Some  researchers  have 
stated  women  signal  submissiveness  by  smiling  more  because  of  learnt  and  internalized 
behaviors  associated  with  a  want  to  please.  Henley  stated  that  a  smile  is  a  woman’s 
“badge  of  appeasement”  [73].  For  example,  embarrassment  and  excuses  are  typically 
accompanied  by  a  smile  [74].  Thus,  less  powerful  individuals  need  to  please,  attend  to 
others’  desires  (often  expressed  by  gaze),  need  to  be  sensitive  and  more  polite  (often 
expressed  by  constrained  body  movements,  taking  up  less  physical  space,  sitting  up 
straight,  etc.)  and  appease  (via  smiling).  This  may  also  echo  verbal  expressions  of 
“positive  politeness”  characterized  by  Brown  and  Levinson  including  exaggerating 
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approval,  intensified  interest,  in  group  identity  markers,  emphasis  on  approval  and 
agreement,  optimism  and  the  use  of  jokes. 

Next  we  will  propose  a  framework  that  integrates  body  behaviors  into  a  model  that  will 
facilitate  the  scoring  of  these  behaviors  and  enhance  the  calculations  of  perceived  face 
threat  based  on  verbal  information  alone.  However,  it  should  be  noted  that  body 
behaviors  are  far  more  complex  and  nuanced,  in  particular  when  cultural  norms  are  taken 
into  consideration.  While  culture-specific  catalogues  for  body  behaviors  will  be 
necessary  to  further  develop  the  model,  it  is  currently  out  of  scope  for  this  effort.  Other 
important  nonverbal  cues,  like  prosody,  proximity,  detailed  facial  analysis,  etc.  have  been 
ignored  in  the  preliminary  model,  which  is  based  on  facial  expressions  (smile,  gaze)  and 
body  posture  alone. 

7 A  Relationship  between  politeness  and  body  behaviors 

As  outlined  in  Figure  1,  politeness  theory  identified  5  different  strategies  with  increasing 
levels  of  ’’redressive  value”,  used  to  mitigate  or  redress  face  threat.  Using  the  example  of 
delivering  a  correction,  one  can  label  this  action  as  1)  stating  the  correction  baldly  on 
record,  with  no  politeness  (“You  are  wrong.”),  2)  displaying  positive  politeness  by 
indicating  solidarity  with  the  hearer  (“I  can  see  why  you  might  think  that  way,  but  you're 
wrong”),  3)  displaying  negative  politeness  by  respecting  the  target’s  freedom  of  action 
and/or  minimizing  the  imposition  (“I’m  sorry  to  tell  you  this,  and  you’re  free  to  disagree, 
but  I  think  you’re  a  little  wrong.”),  4)  stating  the  news  off  the  record  by  using  an  indirect 
method  (“What  about  X  [another  choice]?”)  or  5)  not  conveying  the  message  at  all.  The 
use  of  these  strategies  will  depend  on  the  nature  and  the  context  of  the  interaction,  where 
a  speaker  will  be  more  polite  when  a)  the  relative  power  of  H  over  S  increases,  b)  the 
social  distance  between  H  and  S  increases  and  c)  the  degree  of  imposition  on  H  increases. 
In  this  example,  correction  will  require  more  politeness  because  it  is  imposing  and  it  will 
also  require  more  politeness  when  conveyed  to  a  superior  or  a  stranger. 

While  a  categorization  of  redress  strategies  into  5  different  ways  to  linguistically  offset 
face-threatening  information  is  applicable  in  many  cases,  and  many  of  the  strategies  can 
be  communicated  both  linguistically  and  non-linguistically  (i.e.  being  deferential  or 
apologetic),  some  of  the  linguistic  strategies  do  not  have  nonlinguistic  counterparts  (for 
example  using  passive  voice  to  impersonalize  a  request).  Thus,  while  Brown  and 
Levinson’s  strategies  are  almost  exclusively  designed  for  linguistic  communication, 
governed  by  linguistic  rules,  they  do  not  always  generalize  to  nonlinguistic  domains.  In 
sum,  Brown  and  Levinson’s  organization  of  linguistic  strategies  seems  useful,  but  taking 
nonlinguistic  behaviors/strategies  into  consideration  leads  to  a  different  categorization  of 
politeness  due  to  a  lack  of  one-to-one  mapping  between  linguistic  and  non-linguistic 
behaviors. 

We  have  already  presented  models  for  the  organization  of  nonverbal  behavior  related  to 
understanding  concepts  of  dominance/affiliation/affect  more  generally,  however  a  few 
researchers  have  put  forward  suggestions  for  the  organization  of  nonverbal  behaviors 
within  the  politeness  framework.  For  one,  it  seems  reasonable  to  focus  on  aspects  of 
communication  that  are  part  of  the  immediate  conversation  (facial  display,  posture, 
vocalics,  gaze),  since  these  aspects  seem  related  to  “facework”.  For  example,  Burgoon 
and  Hale  [75]  identified  3  primary  themes  of  relational  communication:  affection- 
hostility,  inclusion-exclusion  and  dominance-  submissiveness.  As  we  will  explain  below, 
these  themes  correspond  with  nonverbal  behaviors  associated  with  the  concepts  of 
positive  and  negative  politeness.  Nonverbal  behaviors  associated  with  relational 
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communication  are  likely  to  impact  face  concerns.  [16] 

Brown  and  Levinson  note  that  positive  politeness  expresses  belonging,  approval,  liking 
as  well  as  affection  and  understanding  [76].  Burgoon  proposed  that  relational  messages 
of  affection  and  inclusion  relate  to  communication  regarding  liking,  positive  feelings, 
rapport  and  affiliation  [77].  Thus,  communication  of  affection  can  be  attributed  to 
positive  face.  In  other  words,  communication  of  inclusion,  belonging  and  liking  addresses 
positive  face  for  acceptance  and  approval.  Various  nonverbal  behaviors  can  be  identified 
as  communicating  positive  politeness,  i.e.  touching,  close  proximity,  ‘smiling’,  direct 
body  orientation  or  ‘postural  mimicry’,  though  such  interpretations  may  (or  may  not)  be 
culturally  dependent.  In  our  example,  saying  “you  are  wrong”  (a  “bald”  statement  and 
verbally  unredressed  and  therefore  probably  rude)  might  be  accompanied  by  a  hugging 
gesture  or  through  mimicking  the  body  behavior  of  H  (both  positive  politeness  gestures 
which  would  reduce  the  threat  by  enhancing  the  positive  face  of  H  and,  therefore, 
reducing  the  D  between  S  and  H). 

In  contrast,  negative  politeness,  relates  to  the  continuum  of  dominance-submissiveness 
because  dominance  is  concerned  with  influence  and  submissiveness  corresponds  -  at 
least  partly-  to  an  awareness  of  imposition.  Submissiveness  operates  to  reduce  face  threat 
by  mitigating  the  asserted  or  implied  P  of  the  S  and  thus,  enhancing  the  relative  P  of  H 
over  S.,  while  dominance  is  concerned  with  displays  of  influence  and  power.  Negative 
politeness  seeks  to  reassure  H  of  his/her  freedom  from  imposition  through  self- 
effacement  and  deference  [1],  which  contrasts  with  dominance  goals.  Deference  or 
submissiveness  can  also  be  communicated  nonverbally  with  smiling  or  facial 
pleasantness — though  these  might  be  nervousness,  hesitation  and  appeasement  (negative 
politeness)  or  social  enfolding,  optimism  and  humor  (positive  politeness).  In  our 
example  from  above,  using  negative  politeness  the  ‘correction’  message  may  be 
expressed  with  nonverbal  displays  such  as  a  hesitant  approach,  tense  position,  smiling  or 
and  postural  complementarity.  Additionally,  there  is  evidence  that  the  visual  dominance 
ratio  is  an  excellent  means  to  understand  dominance  and  submissiveness  behaviors  in 
dyadic  communication  (and  more  recently  also  of  group  behaviors).  See  also  Table  3. 

Other  researchers  have  categorized  nonverbal  behaviors  in  response  to  FTA  in  a  similar 
manner.  For  example,  Ambady  et  al.  [15]  identified  affiliative  strategies  (open,  affiliative, 
joking),  circumspect  strategies  (uncertain,  indirect,  avoidant,  apologetic)  and  other- 
oriented  strategies  (attentive,  concerned,  agreement  seeking,  polite,  approving).  The 
authors  distinguished  affiliative  strategies,  which  consist  exclusively  of  positive 
politeness  strategies.  Cirmcumspect  strategies  consist  of  negative  (uncertain,  avoidant, 
apologetic)  and  off-the-record  strategies  (indirect).  Other-oriented  strategies  consist  of  a 
mix  of  positive  politeness  strategies  that  take  the  needs  of  H  into  consideration  (attentive, 
concerned,  agreement  seeking,  etc.)  as  well  as  negative  politeness  strategies  that 
minimize  imposition.  Their  category  of  other-oriented  strategies  consist  of  a  mix  of  both 
positive  and  negative  strategies,  reflecting  newer  research  suggesting  that  the  delineation 
of  positive  and  negative  strategies  may  not  be  clear  cut  and  that  individuals  tend  to  use  a 
mixture  of  both  types  of  strategies  [78;76]. 

Based  on  the  theories  outlined  above,  we  propose  a  framework  (illustrated  in  Figure  13) 
that  focuses  on  nonverbal  expressions  of  face  threat.  Just  like  Brown  and  Levinson’s  5 
linguistic  politeness  strategies  can  be  linked  to  nonlinguistic  expressions  indicative  of  the 
degree  of  face  threat,  body  behaviors  can  be  organized  along  similar  levels  of 
expressions,  with  degrees  varying  from  high  to  low.  When  face  threat  is  high,  appropriate 
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Figure  13.  Proposed  scoring  model  for  nonverbal  behavior. 

redressive  strategies  need  to  be  chosen  to  counter-balance  the  threat.  This  categorization 
offers  a  different  taxonomy  than  Brown  and  Levinson  with  a  somewhat  different  focus  on 
the  utilization  context  and  resulting  impact,  though  it  would  appear  that  the  exhibited 
behaviors  are  the  same  in  both  cases. 

Using  the  example  of  delivering  a  correction:  if  I  now  say  “you’re  wrong”,  but  I  do  it 
with  a  smile,  in  a  submissive  posture  (i.e.  postural  complementary  to  dominant  posture) 
and  while  looking  away,  the  overall  redressive  score  will  be  very  different  than  when  1 
convey  the  exact  same  message  without  a  smile,  in  a  dominant  posture  (i.e.  postural 
mimicry  in  response  to  dominant  posture)  and  while  looking  straight  at  H. 

When  looking  at  body  posture  alone,  the  information  value  of  a  simple  dominant  or 
submissive  position  is  pretty  clear.  However,  when  looking  at  postural  changes  over  time 
as  a  reaction  to  the  posture  of  S,  then  (we  argue)  social  distance  needs  to  be  taken  into 
consideration.  Based  on  the  literature,  postural  mimicry  would  minimize  threat  if  social 
distance  is  low  (keeping  a  higher  score),  while  postural  complementary  poses  less  threat 
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in  a  relationship  where  D  (S,H)  is 
relatively  high.  If  however  the 
opposite  were  the  case,  threat 
would  be  higher.  In  our  model,  if 
D  (S,H)  is  low  and  postural 
mimicry  is  displayed,  it  would 
count  towards  showing  affiliation 
(which  in  turn  is  likely  to  address 
face  needs)  and  inclusion.  On  the 
other  hand,  if  D  (S,H)  is  low  and 
postural  complementary  is 
displayed  (i.e.  a  dominant  posture 
in  response  to  a  submissive 
posture)  it  can  be  considered  a 
sign  of  dissolution,  or  exclusion 
(see  Figure  14.). 

The  outlined  body  behaviors  in 
this  section  are  sample  candidates 
to  be  integrated  in  an  overall 
cross-cultural  scoring  model.  The 
model  is  far  from  perfect  or 
complete  and  should  be  understood  as  a  preliminary  attempt  to  organize  nonverbal 
behavior. 

8.  Incorporating  Body  Behaviors  into  the  EE 

8. 1  Initial,  Simple  Case 

Body  behaviors  can  be  readily  incorporated  into  the  Etiquette  Algorithm  as  we  previously 
defined  it,  although  there  are  some  representational  and  conceptual  challenges  to  doing 
so.  The  Brown  and  Levinson  model,  while  it  focuses  on  linguistic  expressions  of  polite 
redress,  is  based  on  a  comparison  of  face  threatening  behaviors  to  behaviors  designed  or 
intended  to  mitigate  or  redress  that  threat.  Insofar  as  body  behaviors  fall  into  either 
category  (threat  or  redress),  they  should  be  readily  covered  by  the  dynamics  of  the 
model — and  by  our  computational  implementation  of  it. 

Indeed,  although  our  focus  in  the  initial  Etiquette  Engine  development  was  on  linguistic 
forms  of  politeness,  several  simple  body  behaviors  were  included  in  the  examples  we 
used  in  our  initial  development  and  testing  of  the  algorithm.  For  example,  in  Vignette  1 
in  section  5.5  (and  cf.  Table  1  above),  the  corporal 

•  is  motioned  to  take  a  seat,  which  he  does  before  speaking. 

•  Takes  off  his  hat  before  sitting  or  speaking 

•  Waits  while  the  mayor  continues  to  write  and  doesn’t  speak  until  he  stops,  “puts 
down  his  pen  and  looks  up  at  the  soldier  expectantly” 

In  Vignette  2  (see  Table  2),  the  corporal: 

•  intemupts  the  Mayor  while  he  is  speaking 

•  “perhaps”  puts  his  hand  on  the  Mayor’s  shoulder 

•  Speaks  loudly 


^  exclude  ^ 


^  exclude  ^ 


Figure  14.  Relationship  between  social 
distance  and  postural 
complementary/mimicry. 
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Of  these,  two  were  explicitly  scored  and  factored  into  our  overall  imbalance  computation 
for  these  two  vignettes  (see  Table  1  and  Table  2).  All  of  these  were  the  corporal’s  actions, 
since  we  were  primarily  focused  on  his  behaviors.  These  two  were: 

•  The  waiting  for  an  invitation  to  speak  (Vig  1),  scored  as  potent  negative 
politeness  at  value  60. 

•  Taking  off  his  hat  (Vig  1),  scored  as  an  instance  of  the  negative  politeness 
strategy  of  deference:  50 

An  additional  two  or  three  were  mentioned,  but  not  scored:  the  soldier’s  sitting  in 
Vignette  1,  and  his  interrupting  the  Mayor  and  putting  his  hand  on  his  shoulder  in 
Vignette  2. 

As  a  simple  test  case  and  thought  experiment,  we  will  work  through  enhancing  the 
description  and  coding/scoring  of  body  behaviors  in  these  two  vignettes  and  then 
illustrate  how  they  may  be  incorporated  more  systematically  in  the  Etiquette  Algorithm. 
This  exercise  will  help  us  identify  techniques,  as  well  as  simplifying  assumptions.  In 
subsequent  sections,  we  will  address  these  limitations  in  more  depth. 

In  each  of  the  enhanced  vignettes  below,  body  behaviors  are  underlined.  We  have 
expanded  the  initial  vignettes  described  above  to  incorporate  additional  body  behaviors 
and  to  incorporate  them  more  explicitly. 

Vignette  1A — High  Face  Threat,  High  Redress,  Higher  Believability:  A 
low  ranking  soldier  (i.e.,  a  corporal,  as  indicated  by  uniform  insignia) 
walks  tentatively  (initially  peering  around  the  open  door,  then  upon 

noticing  the  Mayor  engaged  in  work  and  being  noticed  by  him  via  a 

quick  glance,  takes  a  step  into  the  room)  into  the  Mayor's  office  and  the 
Mayor  motions  him  quickly  to  a  seat  with  a  hand  gesture.  The  soldier 
takes  off  his  hat  and  sits  down,  waiting  quietly  in  a  constricted  manner 
with  his  hands  in  his  lap  and  devoting  attention  to  the  Mayor  while  the 
Mayor  continues  to  write  something.  The  Mayor  finishes  up  writing, 
puts  down  his  pen  and  looks  up  at  the  soldier  expectantly.  The  soldier 
returns  the  Mayor’s  gaze  briefly  but  then  averts  direct  eve  contact,  and 

then  says,  “I’m  sorry  to  interrupt  your  work,  Mayor  Fredrickson,  but  my 
name  is  Corporal  Jones  and  I've  been  put  in  charge  of  your  escort  to  the 
event  tonight.  I  was  wondering  if  it  would  be  possible  for  you  to  let  me 
know  where  I  can  meet  your  wife  so  that  I  can  get  her  there  on  time?” 

This  last  utterance  is  accompanied  by  a  palm  up  beat  gesture,  a  smile  and 
an  eyebrow-raised  questioning/beseeching  expression  and  vocal  tone. 

Vignette  2 A — High  Face  Threat,  Low  Redress,  Lower  Believability'.  As 
for  vignette  1  above  except  that  the  soldier  acts  and  speaks  differently. 

Here,  he  strides  into  the  room  without  a  pause  or  acknowledgement, 
coming  all  the  way  from  the  door  to  behind  the  Mayor’s  desk.  He 

interrupts  the  mayor  while  he  is  writing  by  putting  a  hand  on  his 

shoulder.  He  puts  his  other  hand  on  the  desk,  leaning  casually  but 
menacingly  over  the  Mayor  as  he  crosses  one  ankle  over  the  other  in  a 

casual  slouch.  As  the  Mayor  turns  to  face  him,  he  makes  direct  eye 
contact  standing  above  the  seated  Mayor,  smiling  at  him  he  says  loudly, 

“Tell  me  where  I  can  meet  your  wife?” 
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Just  to  make  explicit  our  interpretations/intuitions  about  these  new  versions  of  the 
vignettes:  by  layering  on  the  body  behaviors  (and  calling  them  out  explicitly  in  the  text 
here3),  we  are  emphasizing  them  to  a  degree.  Our  intuitions  are  that  in  Vignette  1  A,  the 
soldier  has  been  made  even  more  tentative  and  perhaps  obsequious,  than  he  was  in  the 
original  version  1 .  Instead  of  a  nominal,  normal,  unremarkable  example  of  average  and 
expected  politeness,  Vignette  1 A  now  begins  to  shade  into  over-polite — probably  not 
unbelievably  so,  but  moreso  than  the  previous  version.  By  contrast,  the  soldier  in 
Vignette  2 A  has  become  positively  menacing.  Instead  of  a  humorous  misalignment  of 
proper  behavior,  2A  now  strongly  encourages  (and  likely  only  makes  sense  under) 
reinterpretation  of  relationships  between  the  Mayor  and  soldier  such  that  the  soldier  is 
asserting  substantially  more  power  than  the  Mayor — perhaps  because  he’s  a  mafia 
member  with  some  inside  dirt  on  the  Mayor  or  something  similar.  With  those  intuitions 
in  mind,  we  will  now  try  to  provide  scores  to  these  body  behaviors  such  that  they  can  be 
incorporated  into  the  equations  described  in  section  5  above.  In  Tables  4  and  5  we 
provide  revisions  to  Tables  1 .  and  2.  to  include  scoring  of  the  body  behaviors  in  the  revised 
Vignettes,  along  with  some  rationale  for  them.  Here,  non-verbal  behaviors  are  shown  in 
green-shaded  rows.  Table  4  illustrates  the  scoring  of  the  added  body  behaviors  we 
included  in  Vignette  1A.  Since  we  did  not  alter  the  P,  D  and  R  relationships  from  those 
used  in  the  initial  Vignette  1,  as  described  in  section  5.5  above,  therefore  the  Wx 
calculation  remains  the  same  and  produces  a  value  of  313.  The  redress  value  (V(Ax))  is 
shown  as  the  summed  total  in  Table 4:  465.  This  means  that  the  calculated  imbalance  for 
this  vignette  is: 


B0:lx=  B0:V(Ax)  -  B0:WX 
lx  =  465  -313  =  152 

Recall  that  the  lx  value  for  Vignette  1  (without  the  added  body  behaviors)  was  -18.  Since 
zero  represents  complete  balance,  and  since  there  is  a  fair  amount  of  variance  on  an 
utterance  by  utterance  basis,  -18  represents  a  very  slight  deviation  from  balance  or 
nominal  etiquette  and  face  relationships,  while  152  represents  a  moderate  deviation 
toward  “over”  politeness.  This  is  very  much  as  predicted  by  our  intuitions  as  reported 
above. 

Table  5  illustrates  the  scoring  of  the  added  body  behaviors  in  Vignette  2 A.  Again,  the  Wx 
calculation  remains  the  same  and  produces  a  value  of  313.  The  redress  value  (V(Ax))  is 
summed  in  the  table:  -325.  This  means  that  the  calculated  imbalance  for  this  vignette  is: 

B0:lx  =  B0:V(AX)  -  B0:WX 
lx  = -325 -31 3  =  -638 


3 

We  suspect  that  explicit  mentioning  of  a  body  behavior  which  would  otherwise  go  without  mentioning  in 
text  (but  would  likely  be  envisioned  by  the  reader)  gives  added  weight  or  salience  to  the  behavior.  For 
example,  the  line  “’I’ll  make  you  pay  for  that  if  it’s  the  last  thing  I  do!’  said  Joe”  may  well  bring  to  mind  the 
body  behaviors  made  explicit  in  the  alternate  line  “’  ’I’ll  make  you  pay  for  that  if  it’s  the  last  thing  I  do!’  said 
Joe,  his  face  suffused  with  blood  and  spittle  flying  from  his  lips.”  But  making  them  explicit  in  the  text 
reinforces  the  image  in  the  mind  of  the  reader  and  may,  therefore,  result  in  still  more  assessment  of  anger  and 
threat.  The  author  literally  found  it  important  enough  to  call  the  reader’ s  attention  to  these  details  by 
explicitly  calling  them  out.  This  inevitable  manipulation  of  the  reader’s  attention  is  an  intrinsic  element  of 
narrative  that  is  not  included  in  free-flowing  and  non-directed  attention  management  and  represents  a 
limitation  of  using  narratives  for  this  type  of  thought  experiment. 
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Table  4.  Redressive  actions  scored  in  Vignette  1 A 


Action  and  Interpretation 

Score 

1 .  The  soldier  “walks  tentatively  into  the  Mayor's  office  Anything  which  startles  another  and/or  which  interrupts 
him/her  from  a  course  of  action  or  topic  of  concentration,  can  be  viewed  as  an  FTA.  Therefore,  any  overt 
attempt  to  avoid  doing  these  things  counts  as  an  act  of  negative  politeness  -  especially  if  it  comes  at  some 
physical,  social  or  temporal  cost  to  the  “Speaker”. 

40 

2.  The  soldier  waits  until  the  mayor  is  finished  and  invites  him  to  speak.  This  seems  to  be  a  very  explicit  form  of 
negative  politeness  (putting  the  other’s  interests  first)  and,  especially  in  this  instance  where  the  H  was  not 
actively  engaged  in  another  conversation,  seems  very  potent. 

60 

3.  The  soldier  takes  off  his  hat.  This  is  a  sign  of  deference  in  our  culture,  which  is  in  turn  a  fairly  potent  negative 
politeness  strategy. 

50 

4.  The  soldier  sits  down,  in  response  to  major’s  hand  gesture.  This  is  a  somewhat  ambiguous  cue  in  context. 

Sitting  minimizing  physical  space  and  readiness  for  physical  action  and  is  therefore  a  negative  politeness 
strategy.  On  the  other  hand,  it  could  also  be  perceived  as  an  invasion  of  private  or  controlled  space  or 
resources —  a  threatening  behavior  in  its  own  right,  which  should  contribute  to  the  FTA  (or  indicate  a  degree  of 
informality  and  presumed  familiarity  which  could  be  modeled  either  as  a  positive  politeness  behavior  or  a 
reduction  in  D).  On  the  other  hand,  in  this  case,  it  is  in  response  to  an  invitation/request  or  order(?)  from  the 
Mayor,  and  therefore  indicates  compliance  and  an  acknowledgement  of  power — a  negative  politeness  strategy. 

In  this  case,  the  compliance  interpretation  to  a  request  seems  to  triumph,  but  because  it  is  an  acknowledgement 
of  power  and  compliance  with  a  direct  request,  it  doesn’t  afford  much  new  power  to  the  Mayor. 

20 

5.  The  soldier  puts  his  hands  in  his  lap.  Another  example  of  minimizing  physical  space  and  readiness  for  physical 
action  and  therefore  a  negative  politeness  strategy,  but  a  fairly  normal  posture  in  context,  so  not  a  particularly 
large  redress  action. 

30 

6.  The  soldier  devotes  his  attention  to  the  Mayor.  This  is  a  fairly  explicit  example  of  attending  to  the  needs  or 
“face”  of  the  other  and  therefore  a  positive  politeness  strategy.  The  fact  that  he  is  doing  this  while  the  Mayor’s 
attention  and  behaviors  are  not  focused  on  him  gives  it  added  potency. 

30 

7.  The  soldier  returns  the  Mayor’s  gaze  briefly  but  then  averts  direct  eye  contact.  This  description  was  intended  to 
indicate  a  focus  of  attention,  but  not  so  intense  or  direct  as  to  be  challenging.  Thus,  it  is  a  further  example  of 
attending  to  the  needs  or  “face”  of  the  other  and  therefore  a  positive  politeness  strategy.  Now,  though  (unlike  6 
above),  the  Mayor’s  attention  is  focused  on  him  therefore  attending  to  the  Mayor  is  somewhat  less  redressive 

20 

8.  The  soldier  apologizes  for  interrupting.  This  is  also  a  negative  politeness  strategy,  though  arguably  a  less 
potent  one  (though  that  may  be  highly  mitigated  by  facial  expressions  and  body  language). 

30 

9.  The  soldier  uses  an  honorific.  Moderately  potent  negative  politeness  strategy. 

40 

10.  The  soldier  poses  the  FTA  as  a  question.  Common  negative  politeness  strategy. 

20 

1 1 .  The  soldier  offers  an  explanation/reason  for  needing  the  information.  Positive  politeness  strategy,  seems 
particularly  powerful  in  this  case. 

35 

12.  The  soldier  appeals  to  the  Mayor’s  (FI’s)  interests.  Positive  politeness  strategy  Powerful  in  this  context. 

30 

13.  The  soldier  is  hesitant  and  skeptical  about  compliance.  A  common  but  reasonably  potent  negative  politeness 
strategy. 

30 

14.  The  soldier  gives  a  palm  up  beat  gesture  and  an  eyebrow-raised  questioning/beseeching  expression  and  an 
eyebrow-raised  questioning/beseeching  expression.  This  reinforces  the  hesitancy  about  compliance 
expressed  verbally  in  13  above  and  represents  additional  negative  politeness. 

30 

TOTAL 

465 

Recall  that  the  lx  value  for  Vignette  2  (without  the  added  body  behaviors)  was  -273. 

This  was  deemed  reasonable  for  a  vignette  designed  and  expected  to  produce  a  large 
imbalance  representing  substantial  impoliteness.  Vignette  2A  was  expected  to  produce 
much  more  extreme  impoliteness  and  it  has,  in  fact,  more  than  doubled  the  imbalance 
score.  This  is  very  much  as  predicted  by  our  intuitions  as  reported  above. 

8.2  Issues  with  the  Simple  Case 

As  expected  and  intended,  the  thought  experiment  with  using  our  Etiquette  Algorithm  to 
more  systematically  and  extensively  incorporate  body  behaviors,  as  conducted  in  the 
previous  subsection,  raises  many  issues.  We  will  identify  these  issues  in  this  section  and 
propose  ways  forward  where  they  are  apparent. 

8.2.1  Body  Behaviors  as  Positive  and  Inverse  Redressors 

We  achieved  the  degree  of  threat  shown  for  Vignette  2A  above  by  treating  the  soldier’s 
aggressive,  threatening  body  behaviors  as  a  form  of  inverse  redress.  While  Brown  and 
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Table  5.  Redressive  actions  scored  in  Vignette  2A. 


Action  and  Interpretation 

Score 

1 .  The  soldier  strides  into  the  room  without  pause  or  acknowledgement.  This  is  interruptive  and  therefore  explicitly 
a  threat  to  negative  face.  Since  the  soldier  offers  little  or  no  redress  for  this  act  (as  he  did  in  vignette  1  A),  this 
increases  the  face  threat 

-40 

2.  The  solider  comes  all  the  way  across  the  room  to  behind  the  Mayor’s  desk.  This  further  enhances  the 

interruption  from  item  1 ,  but  also  constitutes  an  invasion  of  personal  and  more  private  space.  Again,  there  is  no 
redress  offered,  hence  this  increases  the  neqative  face  threat 

-50 

3.  The  soldier  interrupts  the  Mayor.  This  is  the  inverse  of  putting  the  other’s  interests  forward,  but  is  perhaps  less 
extreme  a  threat  than  it  was  a  redress  in  Vignette  1 A  since  the  Mayor  was  not  actively  engaged  with  another 
person. 

-40 

4.  The  soldier  puts  a  hand  on  the  Mayor’s  shoulder.  This  could  (in  our  culture)  be  either  an  extreme  positive 
politeness  behavior  (indicating  strong  familiarity  and  low  social  distance)  or,  in  the  hands  of  those  who  do  not 
have  that  relationship,  a  strong  invasion  of  personal  space  and  a  presumption  of  familiarity  which  would  cause 
face  threat.  Since  there  is  no  evidence  of  such  familiarity  in  the  Mayor’s  mind,  this  would  be  interpreted  as  a 
stronq  threat. 

-60 

5.  The  soldier  puts  a  hand  on  the  Mayor’s  desk.  Another  powerful  invasion  of  personal  space  and  therefore, 
interruptive  and  threatening  to  negative  face.  Perhaps  not  quite  as  potent  as  physical  touching  in  4  above 

-55 

6.  The  soldier  remains  standing  while  the  Mayor  is  sitting.  A  somewhat  mild  assertion  of  power  through  not 

minimizing  personal  stature  and  posture  for  physical  threat.  This  could  be  mitigated  or  even  reversed  by  context 
(e.q.,  the  standinq  servant  who  is  servinq  food  to  the  seated  master)  but  is  not  in  this  context 

-20 

7.  The  soldier  leans  over  the  Mayor.  Another  invasion  of  personal  space,  as  well  as  an  adoption  of  a  posture  of 
physical  enlargement  and  potential  for  physical  action.  In  addition,  this  blocks  the  view  of  the  Hearer,  thereby 
further  threatening  neqative  face. 

-50 

8.  The  soldier  slouches/behaves  casually.  This  is  a  presumption  of  familiarity  or  informality  which  threatens  positive 
face  (though  perhaps  a  culture-specific  one).  By  failing  to  provide  the  degree  of  attention  and  deference  due  to 
the  Mayor’s  position,  but  soldier  threatens  his  expectation  of  being  valued  by  others. 

-20 

9.  The  soldier  makes  direct  eye  contact  while  standing  above  the  Mayor.  Again,  perhaps  a  culture-specific  threat 
and  dominance  display.  Starring  down  another  can  be  a  dominance  of  attention  and,  therefore,  a  negative  face 
threat. 

-30 

10.  The  soldier  is  very  brief  and  therefore,  takes  little  of  the  mayor’s  time.  This  could  be  counted  as  a  negative 
politeness  strategy  of  directness  (albeit  not  a  very  effective  or  unambiguous  one). 

30 

1 1 .  This  could  perhaps  also  be  counted  as  an  example  of  the  positive  politeness  strategy  of  optimism  and  assumed 
compliance — though  again,  not  unambiguously. 

10 

TOTAL 

-325 

Levinson  talk  about  verbal  “politeness”  behaviors  primarily  in  the  context  of  their  use  to 
redress  face  threats,  it  seems  quite  apparent  that  other  behaviors  (whether  verbal  or  non¬ 
verbal)  can  contribute  to  a  perception  of  threat  rather  than  ameliorating  or  offsetting  it. 

Thus  we  used  the  technique  of  simply  inverting  the  sign  and  treating  threatening 
“redress”  behaviors  as  contributing  negatively  to  the  V(Ax)  term  in  the  equation.  This 
has  the  effect  of  increasing,  rather  than  decreasing,  the  negative  imbalance  (lx)  in  the 
vignette. 

While  this  use  of  “impolite”  behaviors  to  increase  the  Face  Threat  in  an  interaction  is  not 
explicitly  discussed  in  Brown  and  Levinson,  it  seems  to  be  a  natural  extension.  After  all, 
verbal  and  non-verbal  human  behaviors  are  not  only  used  to  redress  inherent  face  threat. 
Instead,  we  have  at  least  as  rich  a  repertoire  of  threatening,  insulting,  power  asserting  and 
dominating  behaviors  as  we  do  redressive  ones. 

Furthermore,  this  approach  was  used  (albeit  not  in  the  examples  shown  in  section  5 
above)  in  our  prior  work  with  verbal  behaviors  alone  where,  for  example,  failing  to 
respond  to  a  greeting  with  a  subsequent  greeting  was  modeled  as  an  insult  rather  than  a 
neutral  activity. 

Admittedly,  the  notion  of  “inverse  redress”  invites  some  linguistic  confusion  when  used 
in  conjunction  with  Brown  and  Levinson’s  (and,  indeed,  much  of  cultural 
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anthropology’s)  use  of  the  terms  “negative  face”  and  “negative  politeness”.  Negative 
Face  is  the  desire  to  have  one’s  will  hold  sway;  to  not  be  thwarted.  By  contrast,  Positive 
Face  is  the  desire  to  be  accepted  and  valued  as  a  part  of  a  group  [1,  p.  62], 

Both  are  “Face  Needs”  and  threats  to  either  can  constitute  a  Face  Threat  that  needs  to  be 
redressed  in  Brown  and  Levinson’s  model.  Negative  Politeness,  then,  in  Brown  and 
Levinson’s  usage,  appeals  to  and  more  potently  redresses  the  Hearer’s  Negative  Face  and 
involves  strategies  such  as  minimizing  and  explicitly  apologizing  for  the  hindrance, 
distancing  the  speaker  from  the  imposition  and/or  explicitly  acknowledging  a  debt  from 
it.  It  has  the  effect,  generally,  of  acknowledging  or  enhancing  the  Speaker’s  Power 
Distance  from  the  Hearer.  By  contrast,  Positive  Politeness  appeals  to  the  Hearer’s 
Positive  Face  by  strategies  which  seek  to  reduce  social  distance  and  enhance  the  Hearer’s 
status  as  a  member  of  a  group  with  the  Speaker. 

Thus,  both  positive  and  negative  politeness  behaviors  have  “positive”  redressive  value — 
they  reduce  face  threats,  albeit  in  different  ways.  In  contrast  to  both  of  these,  the  Inverse 
Redressive  acts  we  have  introduced  do  the  opposite — they  increase  face  threat.  Indeed,  it 
is  quite  probable  that  there  are  different  classes  of  inverse  redressive  acts  that  achieve 
their  threatening  properties  by  offending  the  Hearer’s  positive  vs.  negative  face,  though 
we  have  not  considered  this  to  date. 

Unfortunately,  the  use  of  “negative  face”  and  “negative  politeness”  substantially  predate 
this  work  and  we  have  done  the  best  we  can  by  terming  these  behaviors  “inverse”  rather 
than  “negative”  redressive  acts. 

Approach:  Model  redressive  actions  as  capable  of  either  redressing  face  threat 
(“redress”)  or  of  exacerbating  it  (“ inverse  redress”). 

8.2.2  Relationship  Between  Inverse  Redress  and  P,  D,  R  and  C 
While  the  above  strategy  of  modeling  “inverse  redressive  acts”  seems  to  work  well  for 
modeling  body  behaviors  (and  verbal  behaviors)  which  threaten  instead  of  redressing,  it 
works  best  “in  the  moment”  during  a  single  interaction.  There  is  a  more  complex 
relationship  between  the  balance  and  kind  of  face  threat  that  is  allowed  to  stand  in  an 
interaction  and  the  evolving  perception  of  Power  Distance  (P),  Social  Distance  (D), 
Imposition  (R),  and  Character  (C)  terms  which  we  have  only  sketched  to  date. 

As  we  mentioned  in  section  5.5  regarding  Vignette  2A,  extreme  or  sustained  imbalance 
scores  invite  all  involved  (Observers,  Speakers  and  Hearers)  to  rethink  their  a  priori 
assumptions  about  P,  D,  R  and  C  terms.  In  Vignette  2A,  we  see  that  the  initial  notion  of 
the  relative  power  of  a  corporal  and  a  mayor  contrasts  extremely  with  the  perceived 
imbalance  in  the  corporal’s  verbal  and  body  behaviors.  Together,  these  invite  the 
Observer/Reader  to  reinterpret  their  roles,  seeking  some  way  to  make  sense  of  the 
corporal’s  behaviors.  Perhaps,  for  example,  he  is  secretly  a  mafia  member  with  a  hold 
over  the  Mayor? 

More  subtle  examples  occur  everyday  in  our  lives.  Not  every  utterance  has  to  be 
balanced,  but  taken  over  time  and  interactions,  we  establish  a  P,  D,  R  and  C  for  those  we 
interact  with.  If  that  balance  is  not  maintained,  then  one  or  more  of  those  dimensions  for 
assessing  face  threat  (and  resulting  redress  behaviors)  will  shift.  We  modeled  an  example 
of  this  sustained  (though  only  over  a  roughly  10  minute  period)  interaction  in  a  mild 
altercation  between  U.S.  and  Russian  astronauts  on  board  the  International  Space  Station 
in  [5], 
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As  a  heuristic  for  practical  use  of  the  Etiquette  Algorithm,  we  have  generally  regarded 
the  P,  D  and  C  terms  of  the  Wx  equation  as  fixed  within  the  duration  of  a  given 
conversation.  Imposition  (R)  is  a  function  of  the  topics  discussed  and/or  requests  made 
and  therefore  can  shift  with  changes  in  topic,  but  is  largely  fixed  within  a  time  scale  for 
that  topic.  By  contrast,  redress  and  inverse  redress  behaviors  can  have  the  effects  of 
contributing  to  the  perceived  imbalance  and,  therefore,  the  perceived  threat  on  an 
interaction  by  interaction  or  even  utterance  by  utterance  basis.  There  is,  clearly,  a  link 
from  the  “local”  perception  of  threat  in  an  utterance,  and  the  “global”  perception  of  P,  D, 
R  and  C,  but  we  have  not  yet  begun  to  map  this  relationship  or  its  dynamics. 

Approach:  Develop  and  flesh  out  the  theory  of  the  link  between  redress  and  perceptions 
ofP,  D,  R  and  C.  Detailed  research  into  this  represen  tation  is  beyond  the  scope  of  this 
work,  but  we  do  propose  a  theoretical  model  in  section  7. 

8.2.3  Temporal  Unit  or  Time  Frame  for  Body  Behaviors 

Another  challenge  for  incorporating  body  behaviors  in  the  etiquette  framework  we 
previously  developed  for  verbal  interactions  is  knowing  what  the  temporal  unit  of 
analysis  and  representation  should  be.  Whereas  verbal  interactions  have  a  natural  start 
and  end  point,  and  are  naturally  interleaved — with  one  party  generally  talking  while  the 
other  listens — none  of  that  is  necessarily  true  of  body  behaviors  which,  after  all,  may 
occur  over  sustained  intervals  (e.g.,  a  persistent  frown  or  scowl),  may  occur  concurrently 
with  the  activities  of  the  “Speaker”  (e.g.,  a  scowl  triggered  by  and  mostly  concurrent  with 
what  the  Speaker  is  saying)  and/or  may  involve  subtle  second  by  second  (or  even  less) 
changes  which  can  be  interpreted  for  meaning. 

What  kinds  of  time  frames  should  we  model/use?  How  should  we  accrue  redressive 
value  “points”  (whether  normal  redress  or  inverse  redress)  over  time  for  sustained 
behaviors?  How  should  we  model  rapidly  changing  behaviors — especially  in 
circumstances  where  an  Observer  might  interpret  them  not  as  successive  and  different 
redress  maneuvers,  but  as  an  initial  perception  that  was  corrected  “mid-flight”,  roughly 
the  body  behavior  equivalent  of  a  verbal  “stumble”  rapidly  corrected? 

One  approach  would  be  simply  to  monitor  for  variation.  Thus,  when  a  scowl  first 
initiates,  it  incurs  its  requisite  inverse  redress  value  in  the  initial  interval  in  which  it 
occurs.  When  a  new  behavior  is  exhibited,  at  whatever  perceptible  rate  it  changes,  the 
changes  should  be  modeled  as  new  events  with  new  redressive  impacts. 

But  this  in  no  way  accounts  for  sustained  body  behaviors.  Should  a  behavior  which 
persists  over  many  minutes  or  hours  never  have  an  impact  on  face  threat  and  redress  after 
its  initial  observation?  This  seems  in  error.  An  alternate  approach  would  be  to  sustains 
the  value  of  the  behavior  for  all  future  intervals4  in  which  it  is  present  and  unchanged 


4  This  does  not  entirely  solve  the  problem  of  what  to  count  as  an  interval,  of  course.  In 
our  prior  work,  there  was  no  consideration  of  intervals  in  which  there  was  no  speech — 
thus,  in  principle,  it  was  impossible  to  give  or  redress  face  threat  without  speaking.  This 
was  an  acknowledged  simplification.  Here,  we  would  suggest  that  all  intervals  in  which 
the  body  behavior  of  one  individual  is  perceptible  by  another,  threat  and  redress  from 
those  behaviors  should  be  mapped  and  scored.  Any  convenient  interval  could  be  used  for 
mapping  but  it  would  seem  that  most  of  our  intuitive  scoring  is  centered  around  the  time 
period  of  utterances  and  conversational  exchanges,  so  perhaps  a  typical  interval  tied  to 
typical  utterance  lengths  might  be  appropriate — say,  a  few  seconds. 
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But  a  simple  thought  experiment  suggests  that  this  still  leads  to  fairly  coarse  and 
inaccurate  results.  Consider,  for 
example,  two  individuals  in  a  room.  One 
suddenly  begins  to  exhibit  threatening 
body  behaviors  (a  scowl  and  an 
enlargement  of  posture  coupled  with 
direct  eye  gaze)  toward  the  other. 

Clearly,  these  behaviors  should  incur 
some  inverse  redress  when  they  first 
occur  (and  are  detected  by  an  Observer). 

If  they  are  sustained  over  minutes  or 
hours,  however,  the  threat  is  not  simply 
sustained.  Rather,  our  sense  is  that  it 
initially  grows,  but  then  (again,  if 
unchanged )  begins  to  attenuate  and  at 
some  point  (perhaps  with  a  nervous 
laugh),  the  observer  will  eventually  come 
to  regard  it  as  an  aspect  of  the  individual’s  Character  (C)  and  new  baseline  expectations 
are  formed  around  it.  At  an  intuitive  guess  (which  could,  perhaps,  be  validated  by 
research),  the  initially  perceived  value  rises  for  approximately  two  minutes  to  roughly 
twice  its  initial  value  and  falls  for  slightly  longer,  roughly  as  in  a  chi  squared  distribution, 
as  illustrated  in  Figure  15. 

Approach: 

•  Model  die  temporal  unit  of  body  behaviors  around  perceived  changes  in  such 
behaviors. 

•  Model  the  effects  of  a  sustained  body  behavior  at  ~15s  in  tervals  over  period  of 
~5  min  as — or  acquire  better  empirically  documen  ted  distributions  of  sustained 
body  behavior  effects. 

•  Body  behaviors  sustained  over  repeated  longer  intervals  than  a  few  minutes  will 
be  incorporated  into  revisions  to  perceived  P,  D,  R  and  C  as  discussed  in  8.2.2. 

8.2.4  Concurrent  Body  Behaviors 

Threat  (and  perhaps  redress)  can  be  incurred  even  during  intervals  where  person  is  not 
speaking.  This  is  something  that  was  not  represented  in  our  earlier  work  since  redress 
and  threat  were  explicitly  tied  to  verbal  acts  and  it  was  assumed  that  either  two  people 
could  not  be  speaking  concurrently  or,  at  least,  that  if  they  were  they  would  not  be 
attending  to  the  other. 

This  does  not  seem  to  present  a  significant  problem  and  is,  in  fact,  already  addressed  via 
the  recommendations  in  8.2.3  above:  simply  model  the  impacts  of  body  behaviors  either 
as  they  change  or  at  a  intervals  of  a  few  seconds  if  they  are  sustained.  This  should  hold 
for  an  observer  regardless  of  whether  the  observed  person  is  speaking,  or  listening,  or 
doing  neither. 

In  principle,  this  means  that  separate  “streams”  of  threat  and  redressive  value  may  need 
to  be  modeled  on  an  ongoing  basis  for  all  participants. 
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Approach:  Model  the  body  behaviors  as  they  occur  regardless  of  whether  the  actor 
exhibiting  them  is  speaking,  listening  or  neither. 

8.2.5  Sequential  Dependencies 

Some  body  behaviors  have  their  significance  partly  or  wholly  in  the  context  of  behaviors 
that  have  gone  before.  For  example,  physical  mirroring  behaviors  (duplicating  the 
behaviors  of  another)  can  be  indicative  of  bonding  and  social  cohesion  [28],  and 
therefore,  likely,  of  reduced  Social  Distance  in  our  terms.  But  the  significance  of,  say, 
crossed  arms  then  becomes  dependent  on  whether  or  not  the  individual’s  interlocutor 
previously  and  recently  crossed  his/her  arms. 

While  the  prominence  of  such  sequential  dependencies  may  be  greater  with  body 
behaviors  than  they  are  for  utterances,  these  relationships  are  not  entirely  absent  in 
spoken  interactions.  For  example,  the  failure  to  issue  a  greeting  in  an  interaction  takes  on 
more  weight  (more  inverse  redress)  if  the  failure  happens  after  the  interlocutor  has  issued 
one  than  if  neither  person  has  done  so.  This  sort  of  tracking  demands  that  a  historic 
queue  of  such  behaviors  be  kept  and  referenced  when  assigning  values  to  each  new 
utterance.  While  this  is  a  bit  more  demanding  of  any  implementation,  it  is  well  within 
feasibility. 

As  to  what  should  be  the  appropriate  size  or  depth  of  that  queue,  we  are  not  sure,  but  the 
duration  of  the  conversation  seems  to  be  a  reasonable  heuristic. 

Approach:  Maintain  a  history  queue  of  prior  body  behaviors  of  approximately  the 
duration  of  the  conversation  or  interaction  and  in  terpret  the  threat  or  redress  value  of 
actions  in  light  of  prior  actions  in  that  queue. 

8.2.6  Other,  More  Intrinsic  or  Sustained  Aspects  of  Body  Behavior 

Another  issue  to  be  dealt  with  in  this  approach  is  representing  and  handling  the  effects  of 
“intrinsic”  body  behaviors — that  is,  aspects  of  bodies  and  the  behavior  perceived  from 
bodies  that  is  neither  intentional  nor  transitory.  For  example,  body  attributes  such  as  size, 
musculature,  age,  physical  agility  (vs.  obvious  disability)  and  particularly,  gender.  These 
attributes  clearly  affect  perceived  threat  and,  perhaps,  perceived  redress  as  well.  A  large, 
young,  agile,  heavily  muscled  man  who  is  standing  passively  in  a  room  is  nevertheless 
more  “threatening”  (probably  both  physically  and  to  one’s  face)  than  an  elderly,  disabled, 
petite  woman — and  would  likely  tend  to  be  treated  with  more  “politeness”  (i.e., 
redressive  behaviors)  in  most  cultures,  unless  other  knowledge  were  brought  to  bear. 

Such  phenomena  might  be  regarded  as  bodily  attributes  rather  than  behaviors,  per  se,  but 
they  nevertheless  influence  interpretations  of,  especially,  power  but  also  potentially  of 
familiarity  and  imposition. 

Note  that  other,  somewhat  more  transitory  attributes  may  also  be  treated  in  a  similar 
fashion — such  as  clothing,  insignia,  tattoos,  etc. 

Again,  we  note  that  such  attributes  are  not  without  their  analog  in  spoken  behavior  where 
the  pitch,  volume,  accent  and  even  choice  of  language  to  use  and  specific  word  choices 
can  also  signify  intrinsic  power  or  weakness — as  well  as  dimensions  related  to  at  least 
familiarity  such  as  tribal,  clan,  group,  class  or  even  professional  affiliations  (through 
jargon  usage). 

A  partial  solution  to  this  issue  comes  through  the  realization  that  these  are  all  cues  not  so 
much  to  the  face  threat  or  redress  present  in  the  immediate  act  as  in  the  perceived 
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“baseline”  P,  D,  R  and  C  attributes  of  the  interactant  him-  or  herself.  Note  that  in  the 
example  provided  in  sections  5  and  8. 1  above,  we  began  with  an  assessment  of  the  P  and 
D  attributes  of  the  interactants  (i.e.,  the  corporal’s  power  relative  to  the  mayor)  as  well  as 
of  the  “raw”  imposition  of  the  act.  The  redress  (or  inverse  redress)  present  in  the  more 
transitory  verbal  interaction  behaviors  was  then  compared  to  the  face  threat  imposed  by 
those  more  stable  assessed  values.  We  even  noted  (cf.  the  example  story  vignette  in  5.5) 
that  the  rank  of  the  corporal  would  be  as  indicated  by  his  uniform  insignia.  All  this 
means  that  less  transitory  bodily  “attributes”  may  be  treated  as  elements  of  the  scene 
which  enable  a  person  or  character  to  infer  the  P  and  D  (and  perhaps  R  and  C)  attributes. 
They  should  not  be  treated  as  weights  or  mitigating  factors  on  the  redress  or  inverse 
redress  value  of  the  (more  transitory)  behaviors  exhibited  in  the  interaction. 

Of  course,  there  are  two  potential  problems  with  this  approach.  First,  the  difference 
between  a  transitory  and  a  non-transitory  attribute  is  fuzzy  at  best.  Attributes  such  as 
accents,  though  themselves  probably  not  usually  transitory,  cannot  be  observed  until  a 
transitory  utterance  is  provided.  Similarly,  if  the  corporal  puts  on  his  uniform  (and  thus 
the  insignia  becomes  visible)  during  the  course  of  an  interaction,  the  impact  will  likely 
produce  an  adjustment  in  perceived  P  value  and  therefore,  perhaps,  in  the  perceived 
politeness  value  of  the  entire  interaction  to  date.  While  this  may  be  a  problem  for 
computational  implementation,  it  seems  to  accord  with  human  perception  and  implies 
that  an  ongoing,  revisable  assessment  of  P  and  (and  probably  R  and  C)  needs  to  be  a 
feature  of  any  sophisticated  implementation. 

And  that  is  the  second  problem — again  (as  in  8.2.2  above)  we  need  a  data  driven  and 
well-fleshed  out  model  of  how  observations  of  transitory  attributes  and  behaviors  enters 
into  a  more  stable  model  of  durable  attributes  of  an  individual.  This  is  well  beyond  the 
scope  of  this  effort  and  gets  close  to  a  general  theory  of  personality  and  social  interaction, 
if  not  of  overall  perception  and  cognition. 

Approach:  Treat  non-  (or  less)  transitory  body  behaviors  and  attributes  as  perceptual 
cues  to  assess  Power  and  Social  Distance  (and  perhaps  Imposition  and  Character ) — 
thereby  factoring  their  influence  into  the  assessed  Wx  side  of  the  equation— rather  than  as 
factors  influencing  or  altering  the  redress  (or  inverse  redress)  side  of  the  equation — and 
thereby  the  assessed  V(AX)  side. 

8.2.7  Cultural  Differences  in  Body  Behaviors  and  Their  Etiquette  Implications 
A  core  issue,  but  one  that  we  resolved  in  prior  EE  work,  is  that  redressive  behaviors 
differ  from  culture  to  culture — both  in  their  core  meanings  and  in  their  relative  weight. 
Classic  and  obvious  examples  of  this  are  hand  gestures — the  circled  thumb  and  forefinger 
that  signify  “OK”  in  America,  signify  (according  to  unverified  web  sites)  that  the  person 
it  is  directed  at  is  a  “zero”  in  France,  an  asshole  in  Brazil  and  Germany,  and  is  insulting 
to  homosexuals  in  Turkey.  More  subtle  examples  the  relative  weight  and  significance  of 
eye  contact,  proximity,  various  forms  of  casual  touching,  etc. 

The  situation  of  cultural  differences  in  behaviors  and  interpretations  is  certainly  not 
limited  to  body  behaviors.  In  fact,  the  linguistic  variations  in  how  to  say,  for  example, 
“I’m  sorry”  (vs.  “Lo  siento”  vs.  “Je  suis  desole”)  are  so  extreme  that  we  don’t  call  them 
“cultural  variations”  but  rather  “different  languages”  and  we  assume  that  one  is 
essentially  incomprehensible  unless  one  “speaks  the  language”. 

We  had  developed  techniques  to  manage  this  kind  of  distinction  within  our  prior  work  on 
the  EE.  We  used  separate  knowledge  bases  storing  the  culturally  recognized  and  valued 


SIFT-SSIM-Report-v08-fin 


46  of  74 


12/1/2014  8:22:00  PM 


s 


Smart  Information  Flow  Technologies 


threat  and  redress  behaviors  for  an  individual  culture.  We  also  had  the  capability  to 
differentially  weight  the  importance  or  perceived  worth  of  the  different  terms  of  the 
etiquette  equation  (cf.  section  6)  to  represent  more  systematic  differences  between 
cultures  (where,  for  example,  culture  A  weighted  social  distance  more  prominently  than 
power  difference,  while  culture  B  did  the  opposite).  Using  these  methods  we  were  able 
to  represent,  for  example,  the  difference  between  an  American  officer’s  intent  to  show 
deference  by  removing  his  hat  (a  behavior  with  redressive  value  in  American  culture)  but 
instead  to  give  offense  by  leaving  his  sunglasses  on  (a  behavior  with  inverse  redressive 
behavior  in  a  Pashtun  context,  but  with  no  particular  value  or  significance  in  American 
culture)  [4]. 

Since  we  have  already  demonstrated  this  capability  for  both  verbal  and  some  (very 
limited)  body  behaviors,  and  since  we  expect  cultural  variations  in  body  behaviors  to  be 
somewhat  less  than  in  grammatical  speech,  we  do  not  anticipate  a  problem  with  this 
issue.  On  the  other  hand,  the  issue  of  how  to  create  these  knowledge  bases,  and  how  to 
capture  cultural  similarities  and  differences  in  them,  remains  a  large  problem  and  will  be 
discussed  in  8.2.9  below. 

Approach:  Employ  different  knowledge  bases  to  contain  different  cultural  (or  even 
individual )  behaviors  and  their  associated  redressive  or  inverse  redressive  values. 

8.2.8  Modeling  Considerations 

Even  in  the  simple  example  above  it  is  possible  to  see  instances  of  behaviors  (both  verbal 
and  non-verbal)  whose  threat  or  redress  value  is  context  or  sequence  dependent.  In 
Vignette  1  A,  for  example,  the  soldier  sits  in  response  to  a  request/invitation  from  the 
mayor — an  act  indicating  compliance  and  deference  rather  than  invasion  and  presumption 
(as  it  would  have  implied  if  done  without  such  an  invitation).  Similarly,  in  vignette  2 A, 
the  soldier  invades  the  mayor’s  personal  space  repeatedly  without  an  acknowledgement 
or  invitation — without  a  greeting  or  permission.  Since  this  is  the  first  time  they’ve 
encountered  each  other  (at  least  in  recent  time  in  the  context  of  the  vignette),  this  lack  of 
greeting  and  acknowledgement  is  additionally  rude. 

Our  approach  to  handling  these  contextual  and/or  sequential  dependencies  is  to 
incorporate  a  context-specific  model  of  the  series  of  actions  expected  and  permissible 
within  the  an  expected  interaction  context.  This  seems  to  be  similar  to  the  notion  of 
social  “games”  developed  by  Michael  Mateas  and  others  at  UC  Santa  Cruz’s  (UCSC) 
Center  for  Games  and  Playable  Media  as  a  part  of  their  IMMERSE  project.  In  fact,  we 
have  sometimes  referred  to  these  schemas  as  “games”  and  the  individual  steps  within 
them  as  “moves”  in  the  appropriate  game.  Figure  16  (an  extension  of  Hgure6  above)  illustrates 
one  such  template  we  developed  for  an  interaction  involving  asking  for  a  favor  under 
prior  research  with  the  etiquette  algorithm. 

Using  templates  such  as  this  has  the  advantage  that  it  permits  a  reasonable  degree  of 
context  and  sequence  to  be  captured  and  represented  in  the  creation  of  context-  or, 
literally,  template -dependent  values  for  each  behavior  in  the  template.  For  example,  the 
issuing  of  a  generic,  or  specific,  greeting  type  can  be  given  a  value  for  its  occurrence  at 
the  beginning  of  this  sequence  that  might  be  different  if  the  same  phrase  or  behavior 
occurred  in  another  template.  We  used  this  kind  of  sequentiality,  for  example,  to  capture 
the  fact  that  a  greeting  given  as  a  “Greeting_Response”  in  this  template  (that  is,  in 
response  to  a  “Greeting”  action)  was  less  redressive  than  the  same  phrase  issued  in  the 
initial  position — as  the  Greeting  action.  In  other  words,  my  responding  to  you  with  a 
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Good  morning. 
Good  morning. 

m. 


My  name  is 
lam  YYY. 

Pleased  to  meet  you. 

I'm  happy  to  meet  you 
too. 

7.  [Gestures  to  sit] 

8.  [sits] 

9.  like  your  office.  It's  very 
comfortable. 

10. Thank  you! 

11. I've  heard  that  you  are  a 
leader  who  can  get  things 
done. 

12. Sometimes,  I  suppose 
that's  true. 

13.1've  come  to  ask  you  for 
your  help. 

14.What  is  it  you  need? 


10. 

11. 


12. 

13. 

14. 


<Greeting> 

<Greeting_Response> 

<Intro> 

<Intro_Response> 

<Acknowledge> 

<Acknowledge_Response> 

<Invite> 

<Invite_Accept> 

<General_Praise> 


<Thanks> 

<Rationale> 


<Rationale_Concurrence> 

<Request> 

<  Req  uest_Ack&Proceed_In 
vite> 


t 


-f 


Begin  Scenario 

Greet 


■’Pre-Introduction 

Introduction 

■•Recognition 
hankl 


■Compliment 


Pre-Request 


Statement 

^-Request 
End  of  Scenario 


.  Figure  16.  Template  for  the  "Ask  a  Favor"  scenario  or  "game". 


greeting  that  is  just  as  warm  or  polite  as  yours  was  to  me  is  less  redressive  than  your 
initial  greeting  was.  On  the  other  hand,  the  failure  to  provide  a  Greeting_Response  after 
a  Greeting  has  been  issued  could  be  modeled  as  having  a  strong  inverse  redress  (or 
threat)  effect. 


On  the  other  hand,  such  templates  worked  well  in  the  context  of  a  semi-scripted  game  or 
interactive  scenario  where  we  could  exert  control  over  the  steps  the  user/trainee  took  and 
guide  or  steer  them  in  accordance  with  the  storyline.  A  few,  well-crafted 
misunderstanding  actions  could  be  used  to  cover  many  instances  of  the  trainee’s  “getting 
off  the  track”.  It  is  currently  unclear  to  us  how  far  this  template-based  approach  can  be 
extended  to  cover  a  completely  unrestricted  range  of  conversational  and  interaction 
options.  We  are  hopeful  that  the  “social  game”  representations  we  understand  are  being 
developed  for  SSIM  by  UCSC  will  be  more  powerful  than  the  simple  ones  we  have  used 
previously.  One  potential  approach  to  less  restricted  interactions  would  be  to  incorporate 
a  nuanced  plan  recognition  and  planning  module. 

Approach:  Employ  game-like  templates  to  structure  interaction  moves  and  associate 
redress  and  threat  values  or  value  modifiers  with  actions  ( verbal  or  non-verbal )  with  the 
position  in  the  template  where  the  action  is  observed. 

8.2.9  Where  do  we  get  those  values? 

Easily  the  most  significant  practical  question  is  posed  in  the  title  of  this  subsection.  The 
etiquette  and  politeness-based  representation  and  reasoning  approach  we  are  advocating 
here  requires  that  substantial  knowledge  be  represented  in  order  to  arrive  at  an 
expectation  or  decision  about  the  politeness  “balance”  between  redressive  behaviors  and 
face  threat  (itself  based  on  power,  social  distance  and  imposition  perceptions)  that  a 
particular  individual  or  simulated  character  has.  When  that  character  changes,  perhaps  to 
represent  a  different  individual  or  one  with  different  cultural  biases,  or  when  the  context 
changes,  new  knowledge  is  required.  Where  does  that  knowledge  come  from? 

In  section  6.2  above,  we  describe  many  approaches  to  reusing  cultural  knowledge  once 
captured  in  a  format  appropriate  for  using  our  approach.  Figure  provides  some  evidence 
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of  the  scale  up  potential  of  using  our  approach.  While  each  of  these  shows  that  there  is 
an  advantage  to  structuring  knowledge  in  the  format  required  for  this  approach,  neither 
fully  articulates  where  that  knowledge  comes  from,  much  less  how  it  will  differ  when  we 
seek  knowledge  about  how  to  interpret  and  value  body  behaviors. 

Brown  and  Levinson  provided  a  taxonomy  for  abstract  categories  of  linguistic  redressive 
behaviors  that,  they  claim  with  supporting  evidence,  are  culturally  universal  (cf.  Figure 2). 
We  have  used  that  taxonomy  extensively  in  our  work  to  date  with  linguistic  forms  of 
politeness.  This  has  helped  immeasurably  in  recognizing  and  scoring  redressive 
behaviors  in  multiple  scenarios  and,  especially,  cross  culturally.  We  have  also  proposed 
using  it  as  a  prime  component  in  a  more  systematic  and  serious  knowledge  acquisition 
and  representation  effort  in  support  of  politeness  reasoning  in  an  extended  simulation. 

But  is  there  an  equivalent  taxonomy  for  body  behaviors?  Not  that  we  know  of.  Brown 
and  Levinson  provide  some  guidance — anything  that  incurs  costs  and  sacrifices  one’s 
own  face  can  serve  as  a  redressive  behavior  to  others.  Anything  that  appeals  to  positive 
face  is  a  positive  redress  strategy  and  plays  more  to  group  membership  and  reduced 
Social  Distance.  Anything  that  appeals  to  negative  face  is  a  negative  redress  strategy  and 
plays  more  to  power  and  maybe  reduction  of  imposition  and  thus  enhances  H’ s  P  and 
reduces  R.  These  general  principles  can  serve  as  a  guide  for  recognizing  and  scoring 
body  behaviors. 

Furthermore,  specific  behavior  types  in  Brown  and  Levinson’s  taxonomy  frequently  have 
their  analogs  in  body  behaviors.  For  example,  Figure  2  includes  negative  politeness 
strategies  such  as: 

•  Question,  hedge — which  has  a  body  analog  in  tentative  posture  and  gesture  or 
movement  into  a  forum,  or  even  the  more  explicit  shoulder  shrug. 

•  Be  pessimistic — which  has  many  analogs  in  facial  expressions 

•  Minimize  imposition — which  has  physical  analogs  in  the  diminution  of  physical 
presence  through  hunching,  bowing,  lowering  one’s  head,  etc. 

Beyond  this,  we  have  begun  work  toward  a  general  framework  in  section  7,  but  are  far 
from  finished.  That  was  never  a  realistic  goal  for  this  project  and,  indeed,  there  are 
hundreds  of  researchers  working  on  aspects  of  it  across  many  fields  (and  several  who 
worked  toward  it  in  SSIM). 

In  the  meantime,  appropriate  knowledge  bases  for  SSIM  will  have  to  be  constructed  via 
expert  interviews  and  tested  for  their  cross-cultural  validity,  perhaps  with  guidance  from 
the  structures  Brown  and  Levinson  provide  and/or  that  other  researchers  have  identified 
and  suggested — much  as  we  have  worked  toward  in  section  7. 

Approach:  Employ  traditional  SME  interview  techniques  while  seeking  generalizable 
frameworks  for  types  of  body  behavior  from  the  literature  as  we  have  done.  The 
reusability  of  the  politeness  framework  should  help  to  amortize  knowledge  acquisition 
and  representation  costs  and  facilitate  more  rapid  scale  up. 
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9.  A  Worked  Example 

9. 1  Background  Scenario  and  Assumptions 

We  sought  one  or  a  few  specific  examples  of  cross  cultural  interactions  with  rich  body 
behavior  content  to  apply  the  ideas  outlined  in  the  rest  of  this  document  to  as  a 
walkthrough  test  case  similar  to  those  we  reported  in  sections  5.5  and  8.1.  After 
consideration  of  various  alternatives,  and  with  some  belief  that  using  materials  familiar  to 
the  rest  of  the  SSIM  project  would  be  valuable,  we  settled  on  some  of  the  Georgetown 
videos  of  role-playing  military  training  scenarios  wherein  soldiers  interacted  with  role 
playing  (but  culturally  “native”)  translators  and  village  elders.  Thus,  while  staged  events 
for  training  purposes,  these  scenarios  were  intended  to  be  very  representative  of  actual 
military  cross  cultural  interactions  and  were,  in  fact,  instances  of  high  fidelity  training 
exercises  that  soldiers  go  through  to  prepare  for  such  interactions. 

We  chose  to  begin  with  an  analysis  of  “Video  1  Bud  and  Mark  Introductions. mo v”  an 
interaction  in  which 
two  soldiers  and  a 
translator  interact  with 
two  village  elders. 

The  video  is  2 1 
seconds  long  and 
involves  simple 
greetings  and 
introductions  but,  as 
will  be  seen  below, 
contains  some  38 
recognizable  body 
behaviors.  A  still 
from  the  video  is 
shown  in  Figure  17. 

There  are  at  least  9 
different  individuals 
visible  in  this  video,  in 
what  appears  to  be  3 
distinct  groups: 

1 .  two  men  in  tribal  (?)  garb  sitting  in  the  foreground  on  chairs  with  a  small  table 
and  hookah  between  them.  One,  (on  viewer's  left),  is  wearing  a  white  robe  and 
may  be  hatless  and  bald  with  a  fringe  of  what  may  be  gray  hair,  may  be  older. 

We  will  call  him  "WR"  for  White  Robe.  The  other  is  wearing  a  tan  or  gray  robe 
(the  video  is  B&W)  and  a  cap  or  small  turban.  He  appears  somewhat  younger. 
We  will  call  him  "GR"  for  Gray  Robe.  Actually,  we  are  told  that  there  is  a  third 
man  present  at  this  small  table  but  he  is  silent  and  not  visible  in  the  video  due  to 
the  angle  from  which  it  was  shot. 

2.  Three  men  in  military  attire  seated  on  a  carpet  on  the  floor.  The  two  on  the  left 
and  right  have  identical  uniforms,  while  the  center  one  is  somewhat  different 
(including  a  hat).  The  soldier  on  the  viewer's  left  ("LS")  seems  slightly  larger 
than  the  one  on  the  right  ("RS")  but  it  is  otherwise  unclear  who  is  senior  in  rank 
or  age.  Both  appear  blond.  Both  appear  ethnically  unlike  the  two  seated  men  in 
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group  1  and  the  third  soldier  in  the  center  ("CS"),  who  is  smaller  and  may  be  of 
the  same  ethnic  group  (at  least  broadly  speaking)  as  GR  and  WR. 

3.  The  third  group  of  4  men  is  in  the  back  of  the  room  relative  to  the  viewer/filmer. 
These  men  are  sitting  on  tables  or,  in  one  case,  kneeling  on  the  floor.  They  are 
dressed  largely  in  western  style  with  pants  and  long  sleeved  T- 
shirts/sweatshirts/sweaters.  One  is  wearing  what  may  be  a  baseball  cap.  They 
appear  to  be  observers  of  the  main  "action"  in  the  foreground—  whether  affiliated 
with  either  group  or  the  observers/raters  of  this  training  exercise  is  unclear—  and 
will  not  be  considered  further. 

In  general,  the  quality  and  viewpoint  of  the  video  is  such  that  detailed  facial  expressions 
are  impossible  to  detect.  Therefore,  these  will  not  be  considered  in  the  example  below. 
Similarly,  the  audio  quality  on  the  video  is  somewhat  poor,  and  we  did  not  have  access  to 
the  translation  of  those  non-English  utterances  which  can  be  detected.  For  those  reasons 
and  the  fact  that  our  emphasis  was  on  reasoning  about  body  behaviors,  we  have  used 
those  verbal  behaviors  we  can  detect  as  temporal  “milestones”  in  the  scenario,  but  have 
otherwise  not  systematically  scored  the  politeness  value  of  the  verbal  behaviors  in  the 
scenario.  Doing  so,  would  be  exactly  parallel  to  the  examples  provided  from  our  earlier 
work  and  reported  above. 

One  implication  of  not  including  these  verbal  behaviors  in  our  scoring  of  this  scenario  is 
that  we  are  underrepresenting  the  number  (and  therefore,  the  combined  redressive 
potency)  of  the  redressive  behaviors  in  the  scenario.  For  those  reasons,  we  might  expect 
the  scoring  in  the  example  presented  below  to  produce  a  result  indicating  moderately 
undressed  face  threat— suggestive  of  moderate  rudeness — on  the  assumption  that  if  we 
also  factored  in  the  impact  of  verbal  behaviors,  face  threat  and  redress  would  be  more 
nearly  balanced.  On  the  other  hand,  the  same  could  be  said  of  our  past  efforts — that  our 
almost  exclusive  focus  on  verbal  politeness  behaviors  should  have  represented  an 
underrepresenting  of  the  total  amount  of  redress  available  in  the  actors’  behaviors  since 
we  were  not  separately  scoring  their  body  behaviors.  The  fact  that  we  nevertheless  got 
reasonable  values  from  scoring  only  verbal  politeness  behaviors  suggests  that  we  may 
have  been  over  counting  the  redressive  value  of  verbal  behaviors  alone. 

9.2  Assessing  a  Scenario 

9.2.1  Identifying  the  Observer  and  His/Her  Knowledge 

An  important  initial  consideration  in  scoring  the  body  behaviors  in  the  video  scenario  is 
identifying  the  “observer”.  Our  etiquette -based  approach  is  inherently  situated  as  the 
perceptions  and  cognition  of  a  specific  observer  with  a  specific  set  of  knowledge  and 
assumptions  about  the  scene  observed  and  the  participants  in  it.  If  the  observer  changes, 
then  the  knowledge  and  valuation  of  the  phenomena  observed  may  also  change  and 
therefore,  the  perceived  politeness  balance  may  also  change. 

While  we  could  potentially  construct  different  valuations  for  each  participant  in  the 
video,  or  for  any  variety  of  outside  observers  with  varying  a  priori  knowledge  of  the 
scenario,  participants  and  their  attributes  and  behaviors,  in  practice,  we  only  had 
reasonable  access  and  insight  into  one  set  of  observers:  ourselves.  Since  we  had  insight 
into  our  own  knowledge  and  a  priori  assumptions  about  the  scenario,  treating  ourselves  as 
the  modeled  observers  made  a  good  deal  of  sense. 
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In  an  implemented  system,  the  knowledge  we  brought  to  bear  would  be  included  in  a 
culture-  or  even  individual-specific  knowledge  base.  Alternate  knowledge  bases  would 
contain  differing  “rules”  and/or  values  and  could  be  “swapped”  to  represent  the 
perceptions  of  different  individuals.  This  capability  was  illustrated  in  our  prior  work  on 
the  etiquette  engine  [4], 

For  us  as  observers  of  the  current  video  scenario,  this  would  include  knowledge  (or, 
perhaps,  “assumptions/biases”)  such  as  the  following,  albeit  formalized: 

•  Individuals  positioned  higher  in  a  room  or  setting,  or  at  the  head  or  center  of 
tables  generally  have  higher  power 

•  People  who  lead  the  action  (e.g.,  speaking  first)  are  generally  “leaders” — higher 
power 

•  Specific  types  of  clothing  are  indicative  of  cultural  and  organizational  affiliation; 
similar  clothing  indicates  similar  affiliation  and  thus,  reduced  social  distance 

•  Greetings  should  be  acknowledged  and  responded  to;  failure  to  acknowledge  a 
greeting  is  strongly  rude 

•  Palms  pressed  together  combined  with  a  head  bow  is  a  polite,  if  somewhat 
formal,  greeting  gesture  in  some  cultures — and  the  depth  of  the  head  bow  is 
indicative  of  the  degree  of  warmth/sincerity  and  perhaps  of  power 
acknowledgement  intended  in  the  greeting 

•  The  body  position  of  all  5  men  is  roughly  similar  -  none  of  them  displays  neither 
highly  constricted  nor  highly  expanded  body  postures  -  affiliation  seems  to  be 
the  goal.  LS  maybe  displays  most  constrictions,  while  WR  most  expansion.  WR 
also  looks  while  listening. 

•  Two  soldiers  keep  touching  their  heads  almost  synchronous  -  mirroring  can  be 
interpreted  as  in-group  marker. 

In  fact,  all  of  the  phenomena  listed  and  all  of  the  values  associated  with  them  in  the 
subsequent  sections  are  based  on  our  perceptions  as  the  observer  of  this  event. 

We  should  note  that  these  perceptions  were  augmented  by  some  information  reported  by 
a  representative  of  the  Georgetown  individuals  who  observed  and  filmed  the  initial 
interactions  (Aubrey  Logan-Terry).  This  information  included  (based  on  the  textual 
descriptions  she  provided  to  us): 

•  The  individuals  in  the  back  of  the  scene  are  trainers  and  researchers  and  were  not 
considered  part  of  the  scenario. 

•  There  were  three  individuals  playing  the  role  of  “village  elders”  seated  in  the 
front  of  the  scene.  They  are  meeting  in  the  village  of  Sepkati. 

•  These  village  elders  were,  in  the  context  of  the  broader  scenario  of  which  the 
film  clip  is  a  small  portion,  each  from  a  different  village  and  they  are  having  a 
meeting  among  themselves  to  discuss  issues  in  their  villages  (stolen  property, 
bombings,  etc.  due  to  terrorist  activity  in  the  area). 

•  The  training  cadre  arrange  the  scenario  such  that  a  team  of  Soldiers  often  arrive 
at  Sepkati  (the  village  where  this  meeting  is  taking  place)  without  knowing  that 
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the  village  elders  are  already  meeting.  This  is  meant  to  emulate  the  "real  world" 
nature  of  arriving  at  a  village  with  little  intelligence  and  trying  to  figure  out  what 
is  going  on  while  being  respectful  and  not  upsetting  the  villagers. 

•  The  soldiers  in  this  particular  video  were  flagged  as  particularly  good  examples 
because  they  entered  and  sat  on  the  floor  in  front  of  the  three  village  elders. 

•  This  is  likely  the  first  visit  of  these  soldiers  to  this  village,  hence  there  is  no 
familiarity  between  the  soldiers  and  the  villagers 

•  The  elders  know  one  another  but  there  is  some  tension.  Ali,  the  “loudest”  village 
elder,  is  noted  for  his  "high  involvement"  from  an  interactional  style  perspective- 
-often  engaging  in  speech  overlap,  topic  change,  etc.  He,  is  the  elder  for  Sepkati 
and  thus  the  host  of  the  meeting. 

•  The  other  elders  may  suspect  him  of  being  involved  with  the  terrorists  and 
enabling  their  activities  in  the  region— the  Soldiers  don't  know  this  at  this  point  in 
their  meeting/conversation.  ,  There  are  hints  about  tensions  among  the  villages 
that  the  training  cadre  have  planted  at  various  other  sites,  so  the  soldiers  may 
suspect  something,  but  their  main  goal  is  to  just  get  in  and  meet  these  elders  and 
gather  whatever  information  they  can  about  the  terrorist  activities  in  the  area 
while  working  to  build  a  positive  relationship. 

•  Most  of  the  soldier  teams  roll  into  the  villages  in  their  Humvees  expecting  a 
possible  kinetic  experience,  they  don't  often  realize  that  they  are  interrupting  a 
meeting  among  village  elders.  The  village  elders  are  not  expecting  the  Soldiers  to 
arrive.  Often  the  Soldiers  are  brought  to  the  village  elder's  house  by  the  local 
police  chief  and,  depending  on  how  respectfully  they  enter,  the  village  elders 
may  be  offended  that  the  Soldiers  are  there.  Again,  this  team  was  flagged  as  a 
"good  stranger"  example. 

•  This  encounter  is  potentially  very  face  threatening  on  many  fronts.  The  village 
elders  are  wary  of  Americans  who  have  come  to  their  villages  in  the  past  and  not 
helped  much  with  the  terrorism  problem.  As  mentioned  above,  the  Soldiers  are 
(inadvertently)  interrupting  an  in  progress  meeting  among  the  village  elders,  so 
they  offer  more  imposition  on  the  village  elders,  though  they  may  not  know  this. 
Those  who  attempted  to  mitigate  this  by  explicitly  apologizing,  offering  reasons 
for  their  visit,  using  a  "help  frame"  are  most  successful. 

•  There  is  also  the  factor  that  the  Soldiers  are  carrying  weaponry.  Some  Soldiers 
did  their  best  to  have  other  Soldiers  "run  security"  so  that  they  could  focus  on  the 
interaction  with  minimal  need  to  have  weapons  featured  visibly  as  weapons  can 
serve  as  an  inherent  source  of  interactional  trouble. 

•  The  interpreters  in  these  scenarios  tended  to  be  as  "neutral"  as  possible— they 
were  not  meant  to  be  from  local  or  rival  tribes  in  hopes  that  they  wouldn't  serve 
as  a  complicating  factor  in  the  Sodliers'  information  gathering  and  relationship 
building  attempts. 

•  The  Soldiers  are  taught  by  their  interpreters  to  use  the  locally  appropriate 
"Namaste"  gesture  (palms  pressed  together  in  front  of  chest,  bowing  body)  upon 
entering  and  leaving  social  encounters.  Some  Soldiers  did,  some  didn't. 

•  The  hand-to-heart  gesture  that  Mark,  in  particular,  uses  is  more  of  an 
idiosyncratic  gesture  that  is  just  part  of  his  interactional  style.  The  role  players 
mentioned  during  their  stimulated  recall  interviews  with  us  that  Mark  was 
"likeable"  and  one  of  them  even  mentioned  how  he  seemed  sincere  with  his  hand 
gestures,  etc 
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9.2.2  Assigning  P,D,R  and  C  values 

In  order  to  assign  values  for  Power  (P),  Social  Distance  (D),  Imposition  (R)  and 
Character  (C)  for  each  of  the  primary  participants  in  the  scenario,  we  began  with  the  a 
priori  knowledge  we  possessed  about  those  involved  and  their  relationships  and  intent. 

Initial  viewing,  before  being  given  the  Georgetown  knowledge  described  above, 
suggested  intuitively  that  there  were  2  primary  groups — the  three  soldiers  and  the  two 
villagers— with  perhaps  some  subdivisions  within  each  of  them.  The  physical  divisions 
and  groupings  in  the  scene,  along  with  the  clothing,  make  these  groupings  fairly  obvious. 

There  seem  to  be  within  and  between  group  affiliations  and  power  differences  which  we 
will  characterize  first: 

•  Groups  wearing  similar  styles  of  clothing,  and,  perhaps,  being  of  similar  ethnic 
groups  are  likely  to  have  less  social  distance  than  those  wearing  different 
clothing  styles  or  from  different  ethnic  groups,  since  each  of  these  serves  as  a 
marker  of  group  affiliation  or  identity.  This  implies  that  the  D  value  between  LS, 
CS  and  RS  should  be  smaller  than  the  D  between  any  of  them  and  WR  and  GR. 
Similarly,  the  D  value  between  GR  and  WR  should  be  smaller  than  that  between 
either  of  them  and  any  of  the  soldier  group. 

•  Individuals  seated  higher  or  otherwise  elevated  (metaphorically  or,  in  this  case, 
literally)  above  others  can  frequently  be  assumed  to  have  more  power  than  those 
seated  lower.  This  implies  that  GR  and  WR  have  a  P  value  higher  than  that  of 
any  (?)  of  the  soldiers. 

•  Within  the  groups,  speaking  and  speaking  first  are  generally  signals  of  greater 
power — though  this  can  sometimes  be  tempered  if  other  signals  of  deference  are 
observed,  such  as  when  a  speaker  is  speaking  for  a  leader,  though  these  are 
largely  absent  in  this  scenario  except,  possibly,  in  the  case  of  the  interpreter  CS. 
This  would  imply  that  LS  has  a  higher  P  than  any  of  the  other  soldiers,  and  GR 
has  a  higher  P  than  WR. 

•  Within  the  soldier  group,  by  the  same  logic  as  used  to  discriminate  the  two 
primary  groups  from  each  other  initially,  we  would  assume  somewhat  greater  D 
between  CS  and  the  other  two  soldiers  than  between  LS  and  RS.  This  is  because 
minor  variations  in  dress  and  ethnic  appearance  suggest  a  greater  affiliation 
between  LS  and  RS  than  between  either  of  them  and  CS. 

•  Prior  to  learning  of  the  context  of  interrupting  an  ongoing  meeting  from  the 
Georgetown  description,  it  was  not  apparent  to  us  whether  the  meeting  between 
the  soldiers  and  the  villagers  had  been  planned  or  spontaneous  and,  if  planned,  at 
whose  request.  There  is  always  some  imposition  stemming  from  the  intrusion 
into  the  life  of  another  by  any  meeting,  but  there  was  no  a  priori  reason  to  believe 
that  it  was  greater  or  less  than  some  baseline  in  this  case.  On  the  other  hand,  the 
fact  that  it  was  an  interruption  of  an  ongoing  meeting,  especially  a  meeting  called 
to  discuss  problems  of  terrorism,  makes  it  a  greater  imposition  than  we  would 
first  have  thought. 

Given  those  facts  and  their  associated  implications,  and  given  the  rough  scales  for  P,  D 
and  R  proposed  in  5.4  above,  and  the  worked  examples  cited  in  5.5,  we  would  propose 
the  following  initial  values  for  this  scenario: 

•  Power — above  we  provided  an  anchor  point  of  100  for  the  power  that  a  professor 
has  relative  to  a  freshman  student  or  a  parent  over  an  early  teenage  child,  and  we 
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scored  the  relationship  between  a  mayor  and  a  corporal  as  300.  Here,  we  might 
guess  that  the  soldiers  have  more  relative  power  than  the  corporal  in  our  earlier 
scenario  (making  the  power  difference  number  P  lower),  but  the  power  difference 
gap  is  probably  not  as  narrow  as  a  professor  and  student.  So,  somewhat 
arbitrarily,  let’s  assume  that  the  power  difference  between  any  of  the  soldiers  and 
any  of  the  village  leaders  is  ~  1 50.  Then,  using  the  observations  about  power 
above: 

o  P(LS,  RS5) — We  thought  above  that  RS  was  likely  to  be  the  leader  of  the 
soldier  group,  but  that  CS  was  the  lowest  powered  individual  in  that 
group.  RS  is  a  soldier  on  a  mission,  so  he  may  slightly  outrank  LS,  or 
may  just  have  the  leadership  on  this  particular  mission.  This  is  close  to, 
but  probably  slightly  greater  than,  our  anchor  point  on  the  power  scale 
for  the  power  relationship  between  project  manager  and  project  worker, 
which  we  scored  as  10.  So,  let’s  call  this  value  206. 
o  P(RS,  CS ) — This  power  difference  seems  somewhat  larger  than  that 
between  LS  and  RS.  Let’s  call  it  50. 
o  P(LS,CS) — should  be  lower  than  between  RS  and  CS,  but  probably 
larger  than  between  LS  and  RS.  Let’s  set  it  at  35. 
o  P( GR,  WR ) — GR  appears  to  be  slightly  more  powerful  than  WR,  at  least 
in  this  meeting,  but  at  a  guess,  that  power  difference  seems  less  than  LS 
overCS.  Let’s  call  it  10. 

o  P(GR,RS) — Above,  we  claimed  that  the  relative  power  difference 
between  the  two  groups  was  -150.  The  relative  power  difference 
between  the  two  leaders  should  be  perhaps  somewhat  less.  Let’s  call  it 
120. 

o  P(GR,LS) — Since  RS  seems  to  have  slightly  less  power  than  RS,  let’s 
call  the  power  difference  between  GR  and  RS  150. 
o  P(GR,CS) — since  CS  has  still  less  power  than  RS,  let’s  call  the  power 
difference  between  GR  and  CS  1 80. 

•  Social  Distance — our  scale  of  social  distance  uses  anchor  points  (for  American 

culture)  of  -1000  for  deep,  old  friends,  spouses,  etc.,  -100  for  extended 
familiarity  or  reason  to  expect  fairly  friendly  relations  such  as  long  term  work 
colleagues,  moderately  extended  family  members  (aunts,  uncles),  etc.,  and  -10 
for  some  casual  friendship,  acquaintance,  but  not  deep  friendship.  The  positive 
end  of  the  scale  is  reserved  for  strangers  with  no  closeness  but  no  antipathy  (10), 
those  of  differing  race,  socio-economic  status,  etc.  in  a  context  of  mild  mistrust 
(100),  and  those  with  some  definite  social  antipathy  such  as  members  of  the 
military  and  those  known  to  oppose  the  use  of  military  force,  members  of 
different  (and  opposing)  religious  groups,  etc.  The  mayor  and  corporal  were 
given  a  D  of  3 — only  slightly  higher  than  the  midpoint  of  the  scale.  In  this 
scenario,  the  groupings  we  perceive  as  Observer  lead  us  to  suspect  that  the  D 
between  the  two  groups  will  generally  be  higher  than  within  the  groups. 
Knowledge  of  the  potential  antipathy  between  GR  and  WR,  however,  might  shift 


5  That  is,  the  power  that  LS  has  H  has  over  RS  as  S.  The  difference  would  be  reversed  if 
the  roles  were  reversed. 

6  Actually,  to  maintain  the  formal  structure  of  the  equations  presented  earlier  in  this 
report,  we  should  note  that  this  value  is  the  Belief  of  Observer  O  and,  when  it  is  entered 
into  a  Wx  or  lx  calculation,  it  should  be  weighted.  To  reduce  clutter,  we  will  skip  this. 


SIFT-SSIM-Report-v08-fm 


55  of  74 


12/1/2014  8:22:00  PM 


s 


Smart  Information  Flow  Technologies 


this  slightly.  Given  these  prior  examples  and  this  background  knowledge,  we 
would  suggest  the  following  D  values  for  the  scenario 

0  D(LS,  RS) — This  should  be  the  closest  of  bonds  in  the  group  as  these  two 
soldiers  are  from  the  same  organization  (by  uniform)  and  same  cultural 
group.  They  have  probably  served  together,  we  would  assume.  Thus, 
we  would  give  this  relationship  a  D  of  -100. 

0  D(LS,CS) — This  relationship  should  not  be  as  strong  as  the  above,  but 
they  are  still  engaged  in  the  same  endeavor  and  have  presumably  worked 
together  at  least  somewhat  previously.  Let’s  give  it  D=  -10. 

0  D(RS,CS) — Presumably  the  same  as  between  LS  and  CS:  D=  -10. 

0  D( GR,  WR ) — They  are  of  the  same  ethnic  group  and  region  and  have 
similar  jobs,  and  have  presumably  met  previously  and  even  worked 
together  on  past  occasions,  but  we  are  told  (in  the  Georgetown 
background  notes)  that  there  may  be  some  antipathy  between  them. 

Let’s  give  them  D  =  0,  partially  to  reflect  the  unknowns  associated  with 
their  antipathy  for  each  other. 

0  D(GR,LS) — They  are  strangers,  meeting  for  the  first  time,  from  different 
organizations.  Furthermore,  there  is  some  a  priori  antipathy  between 
their  roles.  Let’s  call  this  D  =  100. 

0  D(GR,RS)—As  for  LS:  D  =  100. 

0  D(GR,CS) — since  CS  shares  a  culture  and  language  with  GR,  D  should 
be  less  than  with  the  soldiers.  Nevertheless,  there  is  no  reason  to  assume 
they  have  had  prior  contact  and  insofar  as  there  is  antipathy  between  GR 
and  the  soldiers,  CS  will  likely  share  in  that.  Let’s  call  this  D  =  50. 

•  Imposition—  For  Raw  Imposition  (R),  a  value  of  0  means  essentially  no  or 

negligible  imposition — a  minor  task  or  request  that  one  could  ask  of  pretty  much 
anyone.  Values  greater  than  0  reflect  increasingly  large  impositions,  while  those 
less  than  0  are  indicative  of  requests  that  are  to  the  benefit  of  the  hearer.  Our 
scale  anchor  points  for  nominal  American  culture  included  -1000  for  an 
invitation  to  do  something  deeply  advantageous  to  the  H  (partake  of  food  when 
hungry  and  poor,  to  receive  a  promotion  or  receive  an  honor,  etc.;  -100  for 
something  advantageous  to  H — for  example,  to  come  over  for  dinner  in  the 
future,  to  attend  a  party,  etc.;  and  10  for  an  invitation  to  participate  in  something 
mildly  advantageous — to  have  a  cup  of  coffee  or  a  beer,  to  borrow  a  minor  object 
(book  or  movie),  etc.  At  the  other  end  of  the  scale,  a  value  of  -10  is  characteristic 
of  a  minor  request  of  minor  advantage  to  S  (e.g.,  directions  in  a  neighborhood 
that  H  is  familiar  with);  100  is  characteristic  of  moderate  imposition — borrowing 
a  moderate  sum,  asking  for  a  ride  to  the  airport,  asking  someone  to  watch  your 
kids  for  an  hour  or  pet  for  a  few  days;  1000  is  characteristic  of  a  large 
imposition — borrowing  a  largish  sum,  asking  for  a  largish  time  commitment 
(help  building  a  fence  or  writing  a  book),  etc.  Something  personally  distasteful 
increases  the  imposition  factor.  We  had  previously  characterized  the  imposition 
in  the  corporal’s  request  to  the  mayor  as  10,  in  large  part  due  to  the  assumption 
that  it  was  a  part  of  the  corporal’s  duties.  Given  these  anchors  and  prior  cases, 
we  would  characterize  the  imposition  of  a  normal  greeting  as  comparatively 
small,  and  possibly  even  advantageous  (negative  imposition)  for  H.  On  the  other 
hand,  the  fact  that  this  greeting  is  interrupting  an  ongoing  meeting  serves  to 
increase  the  imposition. 

0  Let’s  characterize  the  general  Rx  (raw  imposition)  of  this  scenario  as  50 
for  the  soldiers  addresing  (and  interrupting)  the  villagers,  and  -10  for  the 
villagers  addressing  the  soldiers  (a  mild  advantage). 
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0  This  “raw”  imposition  should  not  fluctuate  at  least  within  the  “micro¬ 
game7”  of  greetings  and  introductions  exchange.  Once  the  game  shifts  to 
something  else  (e.g.,  asking  a  favor),  then  a  new  raw  imposition  value 
may  be  asserted.  Fluctuations  in  perceived  politeness  and  face  threat 
may  well  happen  within  the  “microgame”,  but  these  will  be  the  result  of 
the  types  of  redress  used  and,  perhaps,  the  presence  or  absence  of 
actions.  For  example,  if  A  greets  B  warmly(e.g.  “It’s  a  great  pleasure  to 
meet  you!”)  and  B  responds  cooly  (“yeah”)  or  not  at  all,  then  B  will  have 
threatened  A’s  face  but  this  is  a  function  of  the  redress  used  and  not  a 
fluctuation  in  the  raw  imposition  of  the  greeting  itself. 

0  Note  too  that  the  “game”  being  played  is  not  the  same  between  all 
participants.  While  there  is  a  “greeting”  interaction  going  on  between 
the  soldier  group  and  the  villager  group,  within  the  two  groups  the 
interactions  are  different  and,  perhaps,  less  well  defined.  Within  each 
group,  there  may  well  be  a  subtle  authority  game  being  played — with  LS 
leading  the  action  and  dictating  phrases  to  be  translated  by  CS,  and  with 
GR  speaking  first  and  for  (?)  the  other  villager(s).  At  any  rate,  since  the 
primary  action  is  the  greeting  game,  and  since  we  don’t  have  much 
information  to  draw  inferences  about  the  other  “micro-games”,  we  will 
focus  on  the  greeting  exchange  exclusively. 

•  Character — As  with  the  mayor  and  corporal  example,  we  know  of  no  specific 

reason  to  suspect  that  any  of  the  individuals  involved,  by  virtue  of  their  character, 
should  be  expected  to  be  prone  to  using  substantially  more  or  less  redress8.  We 
will  treat  this  value  as  0  throughout. 

With  these  assumed  values,  we  now  have  enough  information  to  compute  the 
Weightiness  of  the  perceived  face  threat  for  each  of  the  primary  participants  in  the 
greeting  engagement.  Where  the  core  formula  is: 


7  This  not  entirely  developed  concept  of  a  micro-game  points  to  the  fact  that  we  believe 
we  are  operating  at  a  level  lower  than  the  “social  games”  proposed  by  Shapiro  and  others 
at  IJCSC.  We  view  social  games  as  broader  and,  likely,  composed  of  specific  “micro¬ 
games”  or  perhaps  “moves”  such  as  the  greetings  and  introductions  move  discussed  here. 
As  one  distinguishing  feature,  a  micro  game  may  be  thought  of  as  an  exchange  with 
homogenous  level  of  imposition,  while  a  full  social  game  may  well  involve  multiple, 
shifting  levels.  For  example,  an  “alliance  game”  (as  an  example  of  a  social  game) 
certainly  imposes  constraints  on  how  a  greeting  is  performed,  but  there  are  likely  to  be 
multiple  micro  games  in  a  social  game — for  example,  greetings,  introductions,  small  talk, 
offering  compliments,  offering  favors,  etc. 

8  Actually,  the  claim  from  the  Georgetown  notes  that  Ali  (GR)  is  “high  involvement”  and 
frequently  talks  over  and/or  interrupts  others  might  be  indicative  of  a  Character  term  and 
might  be  justification  for  increasing  the  Wx  term  by  including  a  positive  value  for  C. 

That  said,  we  are  not  sure  that  this  is  the  case  (GR’ s  behaviors  may  well  be  intended  to  be 
brusque  and  somewhat  offensive)  based  on  this  little  data  and,  furthermore,  they  seem 
that  way  to  us  as  observers.  For  those  reasons,  as  well  as  simplicity,  we  will  keep  C  at  0 
for  all  participants.  Similarly,  while  our  equations  permit  the  use  of  alternate  weightings 
for  the  various  terms  in  the  equation,  we  know  of  no  basis  on  which  to  assign  such 
weights  in  this  scenario.  Therefore,  they  will  be  omitted. 
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WX=D(S,H)  +  P(H,S)  + 
Rx  +  C(S) 

Thus,  in  the  greetings  and 
introductions  “micro¬ 
game”  we  obtain  the 
values  for  Wx  as  shown  in 
Table  6.  These  results 
match  our  intuition  that  the 
soldiers  in  general  are 
showing  more  deference 
and  are  attempting  to 
mollify  any  threat  that 
might  be  present,  while  the 
villagers  are  somewhat 
less  concerned  about  that 
(as,  admittedly,  they 
should  be  given  that  we 
created  the  values  for  P,  D, 
and  R  on  the  basis  of  that 


Table  6.  Computations  for  W<  for  the  core  participants  in  the 
greeting  scenario. 


Participants 

S  H 

P(H,S) 

Compnents 

D(S,H) 

Rx 

Total  Wx 

LS 

GR 

150 

100 

50 

300 

GR 

LS 

-150 

100 

-10 

-60 

L5 

CS 

-35 

-10 

CS 

LS 

35 

-10 

RS 

CS 

-50 

-10 

CS 

RS 

50 

-10 

CS 

GR 

180 

100 

50 

330 

GR 

CS 

-180 

100 

-10 

-90 

RS 

GR 

120 

50 

50 

220 

GR 

RS 

-120 

50 

-10 

-80 

LS  leans  toward  GR 

LS  looks  down 

LS  looks  at  CS 

RS  looks  down  (at  notes) 

RS  looks  at  CS 

WR  looks  at  GR 

WR  looks  atCS/RS 

CS  faces  RS 

CS  faces  GR 

CS  looks  at  LS 

CS  scratches  eye 

CS  head  raise  (attentive) 

CS  open  palm  beat  to  RS 

CS  speaks  to  RS  (?  Quiet) 

RS  says  "namaste" 

RS  does  palm  press  gesture 

GR  looks  at  CS/RS 

GR  looks  at  RS  w/  slight  head  tilt  (receptive/atte 
GR  looks  at  CS/LS 

GR  says  "namaste*1  (more  abrupt) 

GR  does  "namaste"  gesture 

itive) 

~ 

RS  smiles  (?)  at  GR 

RS  says  "my  name  is  Matt" 

RS  hand  to  heart 

CS  translates  "my  name  is  Matt" 

GR  repeats  "Matt" 

GR  VERY  slight  head  nod 

RS  looks  at  LS  and  leans  forward/toward  LS 

RS  points  atLS 

CS  leans  back  to  permit  RS  pointing 

RS  “this  is  my  friend" 

CS  “Kadorska" 

CS  open  palm  pointing  to  LS 

LS  says  "namaste” 

LS  hand  to  heart 

LS  leans  toward  GR 

LS  head  nod  toward  GR 

GR  leans  toward  LS 

GR  says  “namaste" 

GR  does  palms  pressed  gesture 

Figure  1 8.  Timeline  of  the  body  behaviors  noted  in  Scenario  1  (Bub  and  Mark  Introductions) — Part  1 . 
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assumption). 


Note  that  we  had  not  previously  implemented  any  techniques  for  enabling  characters  in  a 
simulation  to  dynamically  assess  the  P,  D,  R  and  C  values  for  other  characters  or  human 
users  they  interacted  with.  Instead,  these  values  were  simply  stored  in  a  knowledge  base 
representing  the  beliefs  of  that  individual  character.  Similarly,  we  had  not  developed 


Behavior  10s 

GR  speaks 
IS  leans  toward  GR 
LS  looks  down 
LS  looks  at  CS 

RS  looks  at  GR  ■“ 

RS  looks  down  (at  notes) 

RS  looks  at  CS 
WR  looks  at  GR 
WR  looks  at  CS/RS 

CS  faces  KS  — 

CS  faces  GR  mm 

CS  looks  at  LS 
CS  scratches  eye 
CS  head  raise  (attentive) 

CS  open  palm  beat  to  RS 
CS  speaks  to  RS  ( ?  Quiet) 

RS  says  "namaste" 

RS  docs  palm  press  gesture 
GR  looks  at  CS/RS 

GR  looks  at  RS  w/  slight  head  tilt  (; ■ssjfAi . 

GR  looks  atCS/LS 

GR  says  "namaste"  (more  abrupt) 

GR  does  "namaste"  gesture 

RS  smiles  (?)  at  GR 

RS  says  "my  name  is  Man"  » 

RS  hand  to  heart 

CS  translates  "my  name  is  Matt" 

GR  repeats  "Matt” 

GR  VERY  slight  head  nod 

RS  looks  at  IS  and  leans  forward/toward  LS 

RS  points  at  LS 

CS  leans  back  to  permit  RS  pointing 
RS  "this  is  my  friend" 

CS  "Kadorska" 

CS  open  palm  pointing  to  LS 
LS  says  "namaste" 

LS  hand  to  heart 
LS  leans  toward  GR 
LS  head  nod  toward  GR 
GR  leans  toward  LS 
GR  says  "namaste" 

GR  does  palms  pressed  gesture 


11s  12s  13s  14s  ISs  16s  17s  IBs  19s  20s 


i 


Figure  19.Timeline  of  the  body  behaviors  noted  in  Scenario  1  (Bub  and  Mark  Introductions) — Part  2. 


techniques  for  dynamically  updating  those  beliefs  based  on  interactions  and  experience 
with  an  individual.  These  challenges  are  not  tackled  here  either,  though  we  note  that  the 
types  of  information  reported  in  the  bullet  lists  above  (and,  especially,  as  reported  to  us  in 
the  Georgetown  notes  on  the  scenario),  such  as  physical  groupings,  seating  positions, 
details  of  clothing,  ethnic  appearance,  etc.  are  exactly  the  right  kinds  of  information  upon 
which  to  develop  reasoning  heuristics  that  would  enable  a  simulated  character  to 
dynamically  make  such  inferences. 


9.2.3  Assigning  V(AX)  Redress  Values 

Figure  118  and  Figure  19  presents  a  timeline  chart  which  identifies  relevant  verbal  and  body 
behaviors  in  the  leftmost  column  and  then  shows  the  temporal  interval  in  which  that 
behavior  occurred.  Times  are  approximate  and  roughly  accurate  to  the  nearest  quarter 
second  and  were  derived  from  watching  the  video  in  slow  motion.  Line  colors 
correspond  to  the  different  individuals  involved  in  the  scenario. 


SIFT-SSIM-Report-v08-fm 


59  of  74 


12/1/2014  8:22:00  PM 


s 


Smart  Information  Flow  Technologies 


Behavior  Redress  Value 

Kedp 

Pos  Fate 

Neg  Face  Comments 

GR  Speaks 

leaning  toward,  especially  as  opposed  to  a  space  enhanced  '‘looming"  can  be 
regarded  as  a  sign  of  attentiveness  and  therefore  a  sign  of  positive  redress. 

Especially  at  the  beginning  of  a  "greeting  micro-game”  it  would  be  very  rude  not 

LS  leans  toward  GR 

20 

GR 

X 

to  look  at  the  person  being  greeted. 

While  LS  Is  not  the  primary  speaker  (or  primary  listener)  at  this  point,  his  failure 
to  give  attention  to  GR  while  speaking  is  impolite,  especially  since  GR  is  speaking- 
-as  would  be  suggested  by  the  VDR  Fsperially  given  that  he  is  a  lower  status 
individual,  his  rudeness  is  exacerbated  OTOH,  staring  at  notes  is  still,  plausibly, 
a  functional  pan  of  the  engagement  (as  opposed  to,  say,  engaging  in  another 

Lb  looks  down 

10 

GR 

t 

conversation),  so  the  offense  is  mitigated 

Ibis  brief  shift  of  attention  away  from  the  speaker  (GR),  especially  when 
directed  to  the  translator  (CS)  is  probably  reasonable  and  inoffensive,  looking  to 
the  translator,  especially  if  reasonably  done  to  receive  a  translation  of  what  the 

IS  looks  at  CS 

0 

GR 

X 

speaker  is  saying,  may  be  almost  as  good  as  looking  at  the  speaker  (but  not  quite) 

In  addition  to  the  positive  politneness  reasoning  described  above,  the  VDR 
would  suggest  that  attending  while  the  other  is  speaking  conveys  dominance  of 
the  other,  thus  a  neg  polite  action.  By  extension,  not  attending  to  the  speaker 

Lb  looks  at  CS  while  GR  is 

when  he  is  speaking  Is  an  Inverse  negative  politeness  (an  insult).  We  have 

speaking 

-5 

GR 

X 

represented  this  as  a  separate  score 

RS  looks  at  GR 

15 

GR 

X 

As  for  Lb  above,  a  pos  polite  gesture. 

In  addition  to  the  positive  politneness  above,  the  VDR  would  suggest  that 
attending  while  the  other  is  speaking  conveys  dominance  of  the  other,  thus  a 

RS  looks  at  GR  while  GR  is 

neg  polite  action.  We  have  represented  this  as  a  separate  score  to  provide  a 

speaking 

to 

GR 

X 

slightly  higher  score  than  otherwise 

As  above,  focusing  attention  on  GR  when  his  words  are  being  translated  but  not 

RS  looks  at  GR  while  CS  is 

actually  spoken  (i.e.,  ignoring  the  translator),  seems  particularly  potent  as  a  form 

translating 

15 

GR 

X 

of  neg  politeness  given  the  VDR- based  reasoning  above,  so  we  add  this  score 

RS  looks  down  (at  notes) 

10 

GR 

X 

As  for  LS  above,  an  Inverse  pos  polite  gesture 

RS  looks  at  CS 

0 

GR 

X 

As  forLS  above. 

RS  looks  at  CS  while  GR  is 
speaking 

-5 

GR 

X 

As  forlS  above. 

WR  looks  at  GR 

- 

ignored  for  this  exaample 

WR  looks  at  CS/RS 

ignored  for  this  exaample 

Facing  (and  presumably  looking  at)  an  individual  is  a  similar  form  of 
attentiveness,  giving  positive  polite  redress.  Since  we  are  focusing  on  the 
relationship  between  RS  and  GR  here,  however,  looking  at  someone  01  MLR  than 

CS  faces  RS 

0 

GR 

X 

GR  is  tantamount  to  inattentiveness  to  GR,  albeit  instrumental  attentiveness 

As  for  other  examples  above,  while  this  is  part  of  his  job  (and  therefore  a 
mitigated  insult)  it  is  nevertheless  power-threatening  to  not  get  attention  when 

CS  faces  RS  while  GR  is  speaking 

-5 

GR 

X 

speaking 

CS  faces  GR 

15 

GR 

X 

As  for  RS  facing  GR  above 

CS  looks  at  LS 

0 

GR 

X 

As  for  CS  facing  RS  above 

This  is  at  least  inattentive  and  perhaps  rude  in  the  sense  of  attending  to  ones 
own  bodily  needs  as  opposed  to  the  others.  Scratching  some  body  parts  would 

CS  scratches  eye 

-5 

GR 

X 

? 

be  conventionally  very  rude,  but  we  know  no  reason  to  extend  this  to  the  eye 

In  context,  this  could  be  a  sign  of  increased  attention  or,  on  the  part  of  the 
translator,  an  acknowledgement  of  what  has  just  been  said  and  agreement  to 

CS  head  raise  (attentive) 

BO 

GR 

? 

X 

begin  translating  In  either  rase,  it  is  a  somewhat  more  potent  neg  polite  sign 

This  beat  while  the  CS  is  speaking  may  be  neutral,  but  it  might  also  be  an 
indication  of  emphasis  passing  along  some  nuance  of  j>ower  or  emphasis  that 
GRsaid  As  such  it  would  be  a  mild  form  of  negative  polteness  We're  giving  ft 

CS  open  palm  beat  to  RS 

CS  speaks  to  RS  (?  Quiet) 

10 

GR 

? 

half  points  as  a  result 

RS  say*"namaste'' 

GR 

Figure  20.  Part  1  of  the  redressive  scores  applied  to  the  the  body  behaviors  observed  in  the  scenario— 

along  with  explanations. 

Figures  20,  21  and  22  convey  the  scores  we  have  assigned  to  each  of  the  body  behaviors  we 
noted  in  Figure  18  and  Figure  19  along  with  some  explanation  or  thoughts  about  why  that 
behavior  should  be  scored  in  that  fashion.  In  all  cases,  we  are  basing  our  scoring  on  the 
approach  we  took  for  verbal  behaviors  in  prior  work,  which  was  in  turn  based  on  our 
reading  of  Brown  and  Levinson,  as  augmented  by  the  literature  and  models  we  reviewed 
in  section.  Key  to  this  interpretation  is  the  guidance  provided  by  Brown  and  Levinson 
(and  reported  in  sections  4  and  5.4  above,  as  well  as  partially  illustrated  in  Figure 2)  that 
positive  redressive  strategies  (those  that  are  affiliative,  enhance  positive  face  and 
emphasize  fellowship  and  group  affiliation,  minimize  social  distance,  etc.)  are  generally 
less  potent  than  negative  redressive  strategies  (that  that  are  focused  on  dominance  and 
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Behavior 

Redress  Value 

Recip 

Pos  Face 

NegFace  Comments 

CS  speaks  to  RS  ( ?  Quiet) 

RS  says  “namaste" 

GR 

A  strong  gesture,  perhaps  made  stronger  still  by  accomodation  to  GR's  culture 
(pos  politeness),  of  positive  politeness  (giving  gifts)  and  of  decreased  physical 

RS  does  palm  press  gesture 

60 

GR 

X 

X 

size  and  threat  readiness  (neg  politeness) 

As  for  similar  attentiveness  markers  above;  here  we  are  scoring  exchanges  with 

RS  so  the  split  attention  undercuts  the  value  slightly,  while  the  fact  that  the  split 

GR  looks  at  CS/RS 

10 

RS 

X 

is  for/with  a  translator  undercuts  the  undercutting. 

As  for  similar  attentiveness  during  speaking  (based  on  VDR)  assignments  above, 

GR  looks  at  CS/RS  while  they  are 

slightly  undercut  because  other  gestures  and  interleaved  speech  by  GR  are  going 

speaking 

S 

RS 

X 

on 

GR  looks  at  RS  w/  slight  head  tilt 

As  for  similar  attentiveness  markers  above;  here  the  shift  of  focus  to  RS 

(receptive/attentive) 

20 

RS 

X 

exclusively,  combined  with  the  increased  attentiveness  earn  more  points 

While  this  is  dearly  attentiveness  directed  away  from  RS,  and  might  otherwise 
be  taken  as  lack  of  attentiveness  and  negative  face  threat  (per  VOR),  It  Is 
nevertheless  exactly  where  RS  is  directing  GR's  attention,  therefore  the 
following  of  directed  attention  is  in  fact  a  fairly  potent  surrender  of  negative  face 

GR  looks  at  CS/L5 

20 

RS 

? 

X 

wants. 

GR  says  "namaste"  (more  abrupt) 

RS 

While  this  is  the  same  gesture  as  RS  gives  to  GS  above,  it  is  done  more  briefly  and 
perfunctorally.  In  practice,  this  could  even  be  offensive.  In  fact,  it  is  done  OVER 
RS's  gesture  and  speech,  thereby  preempting  him.  We  will  score  It  negatively  as 

GR  does  “namaste"  gesture 

-10 

RS 

X 

X 

a  result,  with  a  strong  temptation  to  score  it  even  lower 

Per  our  lit  review,  smiles  are  more  likely  to  be  tokens  of  acknowledgement  of 

RS  smiles  (?)  at  GR 

40 

GR 

X 

dominance  and  hence,  moderately  potent  neg  politeness  indicators 

RS  says  ’’my  name  is  Matt 

GR 

This  is,  we  believe,  a  non-formalized  gesture  which  conveys  a  degree  of 
sincerity.  As  such,  it  does  not  incur  deference  points  (neg  politeness)  for 
acceeding  to  the  other's  culture,  but  does  Incur  pos  politeness  points  for 

RS  hand  to  heart 

30 

GR 

X 

friendliness 

CS  translates  "my  name  is  Matt" 

GR 

We  are  sconng  this,  somewhat  tentatively,  as  a  verbal  instance  of  mirroring, 
allbelt  formalized  and  abbreviated  As  such,  in  affords  limited  neg  politeness  to 
RS  since  he  directed  GR's  attention  here  The  fact  that  the  move  is  attenuated, 
combined  with  the  fact  that  it  is  mlmicing  rather  than  complementary  implies 

GR  repeats  "Matt" 

10 

RS 

X 

that  this  is  a  mild  form  of  negative  politeness. 

Again,  a  highly  attenuated  pos  politeness  gesture  indicating  acknowledgement, 
and  deference  (negative  politeness  via  body  posture  minimization).  The 

GRVERY  slight  head  nod 

-10 

RS 

X 

X 

attenuation  of  it,  though,  conveys  dominance  and  may  even  be  more  offensive 

We  are  interested  in  the  effect  on  GR  here,  and  this  serves  as  an  instance  of  RS 

RS  looks  at  LS  and  leans 

"offering  a  gift"  to  GR  in  the  form  of  an  introduction  to  RS.  A  mild  pos  politness 

forward/toward  L5 

20 

GR 

X 

effect 

RS  points  at  LS 

20 

GR 

X 

A  further  effect  of  offering  a  "gift"  to  GR 

This  is  a  fairly  strong  offering  of  deference  (minimizing  one's  position  so  as  to 

CS  leans  back  to  permit  RS 

permit  the  other  to  hold  sway)  from  CS  to  RS.  For  GR,  though,  this  oonvey's  little 

pointing 

10 

RS 

X 

beyond  CS'  recognition  of  RS'  authority. 

RS  "this  is  my  friend" 

GR 

CS  "Kadorska" 

GR 

As  translator,  this  is  C5  conveying  RS'  "gift"  of  the  introduction.  As  such,  it 
conveys  that  gift  (pos  politness)  to  GR,  as  well  as  conveying  RS'  authority  (neg 

CS  open  palm  pointing  to  LS 

20 

GR 

X 

X 

politness) 

LSsays  "namaste" 

GR 

Figure  21 .  Part  2  of  the  redressive  scores  applied  to  the  body  behaviors  observed  in  the  scenario — along  with 
explanations. 


submission,  redress  negative  face  and  seek  to  mollify,  reduce  the  responsibility  for  or 
directly  apologize  and  perhaps  incur  a  debt  for  the  face  threat,  that  minimize  threat  to 
power,  etc.).  We  have  previously  operationalized  this  distinction  as: 

•  The  value  of  the  use  of  an  individual  positive  redressive  strategy  (see  Brown  and 
Levinson,  p.  102,  Figure  3,  for  a  list  of  such  strategies)  will  provide  from  1  to  40 
“units”  of  redress. 

•  The  value  of  the  use  of  an  individual  negative  redressive  strategy  (see  Brown  and 
Levinson,  p.  131,  Figure  4,  for  a  list  of  such  strategies)  will  provide  from  20  to 
60  units  of  redress. 

We  tried  using  that  valuation  here,  though  as  noted,  by  only  scoring  body  behaviors  we 
knew  we  were  undervaluing  the  total  redress.  We  ran  into  four  challenges  however: 
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Behavior 

Redress  Value 

Reap 

PosFace  Neg  Face  Comments 

LS  says  "namaste" 

GR 

As  for  the  namaste  gesture  performed  by  RS  above,  but  minus  a  few  points  for 

LS  hand  to  heart 

SO 

GR 

X  X 

missing  the  culturally  appropriate  gesture 

LS  leans  toward  GR 

20 

GR 

X 

As  above 

LS  head  nod  toward  GR 

30 

GR 

X  X 

more  potent  acknowledgement  and  physical  diminuation  than  GR  did  above 

While  GR's  action  is  directed  at  LS,  It  is  at  the  direction  of  RS,  hence  we'll  score 

GR  leans  toward  LS 

20 

RS 

X 

these  points  for  RS 

GR  says  "namaste" 

RS 

Again,  the  statement  and  esp.  the  gesture  are  abbreviated  relative  to  what  LS 

GR  does  palms  pressed  gesture 

5 

RS 

X  X 

did.  Thus,  we  give  It  fewer  redress  points,  and  might  even  consider  It  offensive. 

Figure  22.  Part  3  of  the  redressive  scores  applied  to  the  body  behaviors  observed  in  the  scenario— along  with 

explanations. 

1 .  The  role  of  multiple  parties — The  Brown  and  Levinson  model,  and  our  prior  use 
of  it,  has  focused  on  the  face  threat  accruing  to  a  single  Hearer  from  a  single 
Speaker.  In  principle,  it  should  be  easy  to  scale  up  using  this  pairwise  approach 
to  any  number  of  participants  in  an  interaction.  There  is  reason  to  believe, 
however,  that  things  aren’t  that  simple — and  this  scenario  illustrates  a  reason 
why.  In  the  course  of  the  introductions  and  greetings,  there  are  multiple 
participants  and  multiple  interactants.  While  WR  never  speaks,  each  of  the  other 
participants  do.  On  the  other  hand,  it  is  reasonably  clear  that  GR  and  RS  are  the 
primary  speakers  and  the  leaders  of  the  interaction,  but  that  each  “represents”  a 
group — RS  the  group  of  soldiers  consisting  of  himself,  CS  and  LS,  and  GR  the 
group  of  villagers  consisting  of  himself  and  (at  least)  WR.  Therefore,  politeness 
or  rudeness  from  anyone  in  either  party  “reflects”  on  the  leader  of  that  group.  In 
this  analysis,  we  were  attempting  to  represent  the  effects  of  body  behaviors  on 
the  computed  politeness  accorded  to  the  two  primary  interactants  (GR  and  RS), 
but  this  required  representing  the  effects  of  the  behaviors  of  the  other  participants 
to  some  extent.  For  example,  when  CS  defers  to  RS  by  leaning  back  to  allow  RS 
to  gesture  to  LS,  we  score  this  as  an  act  of  polite  redress  to  RS.  While  GR  has 
not  “given”  this  redress  to  RS,  it  was  done  in  his  presence  and  therefore  conveys 
redressive  value  to  RS  in  the  context  to  an  outside  observer  (i.e.,  us).  Note  that 
VDR  in  particular,  provides  a  more  thoroughly  realized  approach  to  computing 
the  effects  of  multiple  actors,  as  outlined  in  section  7.1  above.  In  section  9.4 
below,  we  illustrate  the  use  of  VDR  to  calculate  group  dominance  via  gaze  as  an 
exploration  of  this  more  aggregative  approach. 

2.  The  role  of  the  translator — The  presence  of  a  translator  is  novel  for  our 
application  of  the  etiquette  model  and  somewhat  challenging — as,  indeed,  it 
might  be  in  social  interactions  in  general.  The  VDR  discussed  in  7.1  above 
suggests  that  facing  and  gazing  at  a  person  while  s/he  is  speaking  is  a  polite 
gesture  which  affords  redress  and,  usually,  power/dominance  to  that  individual. 
But  in  the  case  of  a  translator,  the  words  being  spoken  are  not  the  translator’s  in  a 
sense,  but  rather  the  speaker  whose  words  s/he  is  translating.  For  this  reason, 
attention  (i.e.,  positioned  toward,  gaze)  to  the  translator  instead  of  the  individual 
who  originally  spoke,  can  seem  less  polite,  especially  if  done  extensively.  It 
would  still,  likely,  be  more  polite  than  doing  something  completely  irrelevant, 
but  not  as  polite  as  looking  at  the  “author”  of  the  initial  words.  Indeed,  in 
another  of  the  Georgetown  videos,  a  different  pair  of  soldiers  is  criticized  for 
precisely  this  failing.  To  handle  this  situation,  we  have  generally  treated 
attention  to  the  translator  as  similar  to,  but  less  potently  redressive  than,  attention 
to  the  original  Speaker. 
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3.  The  role  of  directed  attention — Directing  a  Hearer’s  attention  to  something  is 
itself  a  face  threatening  act — an  impingement  upon  his/her  autonomy  (negative 
face).  Nevertheless,  when  done  at  the  appropriate  time  in  a  Greeting  micro¬ 
game,  we  think  it  serves  not  as  a  threat,  but  rather  as  a  “gift”  (albeit,  an  expected 
one)  of  the  introduction  to  other  participants  in  the  Speaker’ s  team — a  positive 
politeness  action.  Following  the  direction,  then,  is  an  acknowledgement  and 
receipt  of  the  gift — and  act  of  deference  and  (minimal)  obligation  and  therefore, 
an  act  of  negative  politeness.  Particularly  with  regard  to  gaze  direction,  in  this 
context,  it  becomes  not  rude/dominant  to  look  away  from  the  Speaker,  but  rather 
compliant  to  look  where  the  Speaker  is  directing.  We  have  scored  particularly 
GR’ s  actions  using  this  understanding. 

4.  The  appropriate  unit  of  analysis — As  will  be  shown  below,  scoring  the  duration 
and  redressive  value  of  different  body  behaviors  allows  us  to  compute  threat, 
redress  and  Imbalance  on  a  moment  by  moment  basis  at  whatever  temporal 
granularity  is  supported  by  our  detection  mechanism  (in  this  case,  our  quarter 
second  analysis  granularity).  But  it  is  quite  common  for  threat  and  redress  to 
fluctuate  repeatedly  throughout  an  interaction  (cf.  [5])  with  little  or  no  ill  effects. 
We  suspect  that  humans  are  used  to  this  fluctuation  and  pay  little  heed  to  it  as 
long  as  things  “balance  out”  over  the  long  term.  Therefore,  we  suspect  that  a 
reasonable  unit  of  overall  analysis  is  not  the  second-by-second  effects  of 
individual  body  behaviors,  but  rather  the  net  effects  of  such  behaviors  (and  the 
associated  speech)  taken  over  at  least  a  “micro-game”  as  discussed  above.  We 
will  illustrate  effects  at  both  the  second-by-second  level,  and  at  the  whole  micro- 
game  scenario  level  in  the  discussion  below. 

We  presume  that,  in  an  implemented  system,  values  such  as  those  in  Figure  20  through 
Figure  22  will  be  represented  in  a  knowledge  base  corresponding  to  each  individual.  As 
for  the  P,D  and  R  values  in  the  previous  section  above,  it  is  expected  that  beliefs  about 
the  redressive  value  (if  any)  of  any  given  body  behavior  would  have  to  be  represented 
for  each  individual  separately — though  there  would  be  cultural  similarities  between 
individuals  from  the  same  culture — and  would  represent  that  individual’ s  beliefs  about 
the  redressive  potency  of  various  acts. 

We  have  admittedly  done  nothing  (beyond  providing  an  initial  and  very  partial 
taxonomy)  toward  developing  techniques  for  recognizing  such  body  behaviors  from  users 
of  a  simulation  system,  nor  for  generating  them  from  characters  within  the  game.  This 
was  not  assumed  to  be  our  charter,  though  it  represents  a  substantial  challenge  in  its  own 
right. 

Finally,  it  is  interesting  to  note  that  the  bulk  of  the  redressive  actions  used  in  the  scenario 
emphasized  positive  politeness.  While  we  did  not  consider  this  beforehand,  in  retrospect 
it  makes  sense:  a  Greetings  and  Introductions  interaction  is  generally  mostly  about 
enhancing  fellowship  and  removing  barriers.  While  power  dynamics  will  certainly  be 
involved  as  well  (as  they  are  in  this  clip),  it  is  perhaps  not  surprising  that  the  bulk  of  the 
moves  are  targeted  at  positive  face  and  the  dimension  of  affiliation. 

9.2.4  Computed  Imbalance  Scores 

Since  the  Wx  scores  computed  for  Face  Threat  in  section  9.2.2  above  were  scored  for  the 
scenario  as  a  whole — what  might  be  called  the  “Greetings  and  Introductions”  micro¬ 
game — the  redressive  value  V(AX)  can  be  examined  either  on  a  time  unit  by  time  unit 
basis  or  taken  over  the  scenario  as  a  whole.  As  discussed  above,  we  believe  that  for  most 
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Temporal  Interval  (in  sec) 


■SUM  of  redress  accorded  to  GR  SUM  of  redress  accorded  to  RS 

Figure  23.  Summed  redressive  value  accorded  to  GR  and  RS  during  each  temporal  interval  of  the 
scenario. 

purposes,  aggregating  redress  over  at  least  a  full  micro-game  will  be  appropriate,  but  for 
momentary  interaction  dynamics  (such  as  might  be  detectable  via  physiometric  or 
neurophysiological  techniques),  tracking  the  momentary  politeness  fluctuations  might  be 
more  appropriate. 

At  any  rate,  the  representation  and  scoring  techniques  we  have  developed  support  either 
approach.  Figure  23  presents  the  momentary  values  of  redress  directed  at  both  GR  and  RS 
throughout  the  scenario.  These  values  were  computed  by  “spreading”  the  redressive 
value  scored  for  each  body  behavior  act  (as  shown  in  Rgures  20-22)  over  the  time  interval 
for  that  act  (as  shown  in  Figure  118  and  Figure  19)  and  then  summing  the  redressive  values 
directed  at  that  individual  over  the  acts  happening  concurrently  within  that  interval. 

Figure  23  makes  it  clear  that  substantially  more  redressive  value  is  accorded  to  GR  during 
most  temporal  intervals  during  the  scenario  than  is  accorded  to  RS.  Since  (as  indicated  in 
Table  6  above),  the  weightiness  of  the  Face  Threat  (Wx)  from  all  of  the  soldiers  to  GR  is 
positive  for  the  scenario  as  a  whole,  while  the  weightiness  of  the  threat  from  GR  to  each 
of  the  soldiers  is  negative,  we  would  expect  the  soldiers  to  “work  harder”  to  offset  the 
threat. 

We  also  note,  however,  that  the  larger  amounts  of  redress  are  applied  later  in  the 
interaction — from  time  8-12.5  and  from  13.75  to  19.25  when,  roughly,  RS  begins  his 
“Namaste”  greeting  and  gesture,  and  then  introduces  himself  and  his  fellow  soldier. 
Earlier  in  the  scenario,  when  GR  is  speaking,  there  is  little  that  RS  can  do  to  increase  the 
redress  other  than  listen  attentively — an  action  that,  in  fact,  is  undermined  to  some  extent 
by  his  partner  (LS)  paying  more  attention  to  his  notes  than  to  GR. 

We  note  also  that  the  overall  sum  of  redressive  body  behaviors  scored  in  this  scenario 
never  exceeds  155  for  GR  and  70  for  RS.  This  is  in  contrast  to  the  Wx  face  threat  values 
we  computed  of  280  when  GR  is  addressed  by  RS,  and  -140  when  GR  speaks  to  RS  (cf. 
Table  3.  ).  In  other  words,  if  we  were  to  compute  Imbalance  (Ix)  of  threat  to  redress  on  a 
moment  by  moment  basis,  we  would  be  left  with  gross  negative  imbalance  (rudeness) 
toward  GR  and  somewhat  positive  imbalance  (over  politeness)  directed  at  RS. 

This  is  one  motivation  for  looking  instead  at  the  overall  redressive  value  directed  at  each 
individual  throughout  the  “micro-game”  scenario.  When  we  score  the  value  of  each 
behavior  once  (regardless  of  the  behavior’s  duration),  and  then  sum  the  values  over  the 
full  scenario,  we  arrive  at  the  following  V(AX)  values: 
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•  Redressive  value  directed  at  GR  (by  RS  and  his  team):  365 

•  Redressive  value  directed  at  RS  (by  GR):  80 


Given  the  initial  Wx  scores,  we  arrive  at  overall  Imbalance  scores  (V(AX)-WX  =  Ix)  of: 


•  For  GR:  365  -  220  =  145 

•  For  RS:  80 -(-80)=  160 


That  is,  from  our  outside  Observer  position,  and  given  what  we  knew  about  the 
relationship  a  priori  (as  represented  by  how  we  scored  P,D,R,C  and  the  V(Ax)  redressive 
acts),  we  conclude  that  both  sides  were  being  fairly  “polite”  to  each  other  (mildly  positive 
Imbalance  scores).  Furthermore,  they  were  doing  so  at  approximately  equal  levels. 

Keep  in  mind  that  these  represent  only  the  body  behavior  scores.  Adding  in  scoring  of 
the  verbal  behaviors  and  the  scores  would  rise — perhaps  double.9 

This  is  in  no  way  to  say  that  the  two  groups  used  approximately  equal  numbers  or  value 
of  polite  behaviors  toward  each  other.  In  fact,  the  soldiers  clearly  used  substantially 
more,  and  higher  potency,  redressive  behaviors  throughout.  But  the  fact  that  their  visit 
was  an  imposition,  especially  on  GR,  meant  that  in  order  for  them  to  achieve  the  same 
level  of  redress  that  GR  had,  they  needed  to  work  much  harder. 

Furthermore,  given  what  we  know  about  GR  and  his  attitudes  toward  RS  (or,  rather, 
American  soldiers  in  general),  and  given  that  it  was  his  meeting  that  was  being 
interrupted,  we  perceive  him  as  acting  fairly  positively  in  this  encounter.  This  is  not  to 
suggest  that  RS  would  necessarily  see  it  that  way,  especially  without  knowledge  of  the 
interrupted  village  meeting. 

9.3  An  Alternate  Scenario 

Since  the  power  of  the  etiquette -based  scoring  approach  can  sometimes  be  better 
appreciated  in  contrast,  we  constructed  a  variant  of  the  above  scenario  used  above.  While 
this  scenario  was  not  actually  observed  or  recorded  in  the  Georgetown  videos,  it  is  based 
on  one  that  was.  The  interaction  which  observers  reported  as  the  worst  of  the  three  they 
observed  involved  soldiers  who  spent  very  little  time  either  oriented  toward  the  villagers 
by  body  posture  or  having  their  gaze  focused  on  the  villagers — even  when  the  villagers 
were  talking.  Instead,  they  oriented  toward  the  interpreter  (our  CS)  throughout.  See 
Figure  24  for  a  still  image  from  this  scenario. 

A  portion  of  this  interaction  was  actually  recorded  and  made  available  to  us,  but  it 
involved  a  temporal  period  slightly  later  in  the  interaction  than  the  “Matt  and  Bud” 
scenario  analyzed  above — a  period  in  which  RS  was  answering  the  question  “why  did 
you  come  to  our  village?”  This  represents  a  move  into  a  different  (and  less  formalized) 
“micro-game”  and  would  have  represented  substantially  more  analysis  effort  on  our  part. 


9  This  may  be  an  argument  that  we  are  according  too  high  a  score  to  the  redressive  value 
of  actions.  If  we  are  to  move  toward  a  system  that  counts  both  physical  and  verbal 
politeness,  it  may  well  be  desirable  to  adjust  the  scales  so  that  summing  physical  and 
verbal  politeness  retains  the  desirable  feature  of  centering  on  zero.  More  test  cases  would 
have  to  be  examined  before  making  that  recommendation  more  strongly,  however. 
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Also  worth  noting  is  that 
this  video  involved  a 
female  soldier  (“Lt. 

Jill”)  in  the  role  of  RS. 

Since  our  time  was 
limited,  and  since  we 
wanted  a  direct 
comparison  between 
alternate  versions  of  the 
“Greetings  and 
Introductions”  scenario, 
we  constructed  a 
hypothetical,  alternate 
version  of  the  scenario 
we  analyzed  previously 
by  simply  altering  some 
of  the  redressive  actions 
taken,  or  not  taken. 

This  was  roughly 

equivalent  of  projecting  the  behavioral  patterns  of  “Lt.  Jill”  into  the  verbal  interactions  of 
the  Matt  and  Bud  from  the  scenario  above.  In  other  words,  we  had  Lt.  Jill  and  her  team, 
as  well  as  the  villagers,  say  the  same  things  that  were  said  in  the  scenario  above,  but 
instead  of  keeping  gaze  and  posture  focused  on  GR,  they  tended  to  orient  toward  the 
translator,  CS,  instead.  Furthermore,  also  in  keeping  with  Lt.  Jill’s  behavior  patterns 
(which  were  physically  very  still  and  non-expressive),  we  removed  the  use  of  the  palms 
pressed  “Namaste”  gesture  and  the  hand  to  heart  gestures. 

Finally,  we  took  advantage  of  the  fact  that  Lt.  Jill  was  female  to  illustrate  another  feature 
of  our  approach.  While  we  do  not  actually  know  how  GR  feels  about  females  or  female 
soldiers,  Hofstede  [29]  reports  that  Iraq  is  fairly  high  on  his  scale  of  Masculinity- 
Femininity  and,  therefore,  it  is  not  unreasonable  to  assume  that  a  male  village  leader 
might  feel  increased  imposition  from,  or  increased  power  difference  over,  a  female  visitor 
interrupting  a  meeting  he  was  conducting.  We  can  model  this  feeling  by  a  revision  to  the 
P,D,R,  and  C  attributes  which  yield  our  Wx  face  threat  weight.  For  our  alternate  scenario, 
we  represented  this  factor  by  changing  the  P(H,S)  from  its  value  in  the  prior  scenario 
(120  for  GR  as  H  and  -120  for  RS  as  H — see  Table  3.  )  to  the  same  value  used  for  CS: 

1 80  for  GR  as  H  and  - 1 80  for  RS  as  H.  This  meant  that  the  Wx  value  for  RS  speaking  to 
GR  was  280  (as  opposed  to  220  in  the  earlier  scenario)  and  for  GR  speaking  to  RS,  it  was 
-140  (as  opposed  to  -80  previously).10 


10  We  should  emphasize  that  we  use  this  example,  based  on  a  real  training  interaction, 
only  to  illustrate  how  some  gender  effects  can  be  captured  in  our  representation.  Gender 
differences  may,  in  fact,  make  achieving  polite  (or  powerful)  interactions  more  difficult 
in  some  cultures — as  illustrated  in  this  example,  but  this  is  no  reason  in  and  of  itself  not 
to  attempt  them.  Furthermore,  “Lt’  Jill’s”  poor  gaze  and  orientation  skills  have  no 
association  with  her  gender,  and  could  just  as  easily  have  happened  to  a  male  trainee. 
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Figure  25  shows,  in  red,  the 
changes  to  the  initial  scenario 
that  were  made  in  this 
alternate  scenario.  All  other 
behaviors  were  included  at 
their  initially  scored  values. 

When  these  changes  were 
made,  the  resulting  moment- 
by-moment  redressive  graph 
is  shown  in  Figure  26.  By 
comparing  this  figure  to  Figure 
23  above  it  should  be 
immediately  apparent  that  the 
redress  accorded  to  GR  is 
generally  not  as  high,  does 
not  reach  as  high  a  peaks,  and 
is  more  frequently  below  that 
accorded  to  RS  than  it  was 
for  the  previous  scenario. 

Perhaps  more  telling  is  the 
overall  redressive  value 
directed  at  each  individual 
throughout  this  alternate 
scenario.  These  values  were: 


Behavior 

Redress  Value 

Recip 

Pos  Face  Neg  Face 

GR  Speaks 

LS  leans  toward  CS  (not  GR) 

0 

GR 

X 

RS  looks  atCS  (not  GR) 

RS  falls  to  look  at  GR  while  GR  Is 

0 

GR 

X 

speaking 

-5 

GR 

X 

RS  looks  at  CS  (not  GR)  while  CS 
is  translating 

CS  speaks  to  RS  (?  Quiet) 

0 

GR 

X 

RS  says  "namaste" 

GR 

RS  fails  to  do  palm  press  gesture 

0 

GR 

X 

X 

GR  says  "namaste"  (more  abrupt) 

RS 

RS  says  "my  name  is  Matt" 

GR 

RS  fails  to  do  hand  to  heart 

0 

GR 

X 

CS  translates  "my  name  is  Matt" 

GR 

GR  repeats  "Matt" 

10 

RS 

X 

RS  "this  is  my  friend" 

GR 

CS  "Kadorska" 

GR 

LSsays  "namaste" 

GR 

LS  fails  to  do  hand  to  heart 

0 

GR 

X 

X 

LS  leans  toward  CS  (not  GR) 

0 

GR 

X 

LS  head  nod  toward  CS  (not  GR) 

LS  fails  to  look  at  GR  when  GR  is 

0 

GR 

X 

X 

speaking 

-5 

GR 

X 

GR  says  "namaste" 

RS 

Figure  25.  Changes  to  the  representation  of  the  “Matt  and  Bud” 

scenario  that  were  made  to  score  and  analyze  our  alternate  scenario 

representing  a  less  polite  interaction. 

•  Redressive  value  directed  at  GR  (by  RS  and  his  team):  105 

•  Redressive  value  directed  at  RS  (by  GR):  80 


The  previous  values  were  365  and  80  respectively,  so  the  lack  of  orientation  and 
attention,  as  well  as  the  omission  of  the  gestures  substantially  reduced  the  redress  given 
to  GR.  Furthermore,  given  the  initial  Wx  scores,  the  overall  Imbalance  scores  (V(AX)-WX 
=  Ix)  are: 
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Fig.  27.  VDR 
Timeline. 


•  For  GR:  105  -280  =  -175 

•  ForRS:  80  -  (-140)  =  220 


These  values  were  145  and  160,  respectively,  previously. 

That  is,  the  omission  of  the  polite  gaze  and  gestures  from  RS  and 
her  team  (combined  with  tension  over  the  presence  of  a  female 
soldier  combined  to  turn  the  interaction  with  GR  from  a  mildly 
positive  one  to  a  mildly  negative  one  in  this  hypothetical  scenario. 
Furthermore,  instead  of  reaching  approximate  parity  in  accorded 
politeness  levels,  GR  is  now  perceived  as  being  quite  polite  to  RS, 
while  RS  has  been  fairly  rude  to  GR.  More  importantly,  this 
simple  example  serves  to  illustrate  how  our  representation  and 
etiquette  scoring  approach  can  be  sensitive  to  minor  variations  in 
body  behaviors — resulting  in  substantial  modifications  to  the 
perceived  politeness  of  the  outcome. 

9.4  Calculating  the  VDR  using  SSIM  training  video 

As  noted  above,  one  of  the  weaknesses  of  our  approach,  which  we 
are  only  partly  overcoming  in  the  above  examples,  is  that  it  is 
geared  to  compute  the  relationship  between  one  Speaker  and  one 
Hearer.  In  practice,  multiple  individuals  may  be  involved  in  an 
interaction  even  if  they  are  not  actively  speaking,  and  their 
behaviors  will  and  should  have  an  impact  on  the  perceived 
politeness  (and  power,  familiarity  imposition,  character  and 
redressive  value  dimensions  which  comprise  it)  which  results. 

This  is  particularly  true  of  body  behaviors,  which  as  we  noted 
above,  may  go  on  contiguous  with  speaking. 

The  Visual  Dominance  Ratio  described  in  section  7.1  above  is  an 
approach  we  learned  about  from  the  literature  for  computing  either 
a  pairwise  or  a  groupwise  gaze  ratio  which  has  demonstrated 
correlates  with  perceived  dominance.  We  took  the  step  of 
applying  it  to  the  same  scenario  we  analyzed  above  (in  section  9.2) 
to  see  if  it  provided  parallel  results. 

Based  on  the  sample  scenario  described  in  section  9,  we  calculated 
visual  dominance  ratios  for  the  four  main  actors  in  the  “Matt  and 
Bud  Introductions”  video.  To  calculate  the  VDR  for  each  person, 
we  analyzed  the  video  1  and  identified  variables  for  “looking  while 
talking  (lwt)”,  “looking  away  while  talking  (talking  away)”,  “total 
time  talking  (talking)”,  “looking  while  listening  (lwl)”,  “looking 
away  while  listening  (looking  away)  and  the  total  amount  of 
“group  talk”  (minus  the  person  for  who  the  ratio  is  being 
calculated),  not  listed  in  this  table.  These  intervals  are  presented  in 
the  timeline  in  Figure  27.  The  VDR  was  calculated  for  4  out  of  5 
visible  actors  in  the  video,  using  Formula  2  in  Table  3  above  (based 
on  Koch,  [43]).  It  reflects  the  visual  attention  that  a  person  A  pays 
to  all  team  members  dependent  on  A’s  and  the  group’s  speaking 
mode  (i.e.,  A  talks  or  A  listens  while  team  members  are  talking). 
The  numerator  consists  of  the  proportion  of  the  entire  speaking 
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Table  7.  Visual  Dominance  Ratio  for  4  group  members. 


GR#wt  4.8  RSfttwt  3 

GRttalking  4.8  RSttalking  4.5 

GRttwl  15  RSttwl  10.75 

GR#ooking(6way  0  RStfookingteway  6.25 

GRttalkingttway  0  RSttalkingtfeway  1.5 

GRWDR _ 1/1.090  RSWDR  ^0.33/0.320 

VDR _ 0.91667  VDR _ 1,040741 

time  of  A  that  A  looks  at  any  group  member.  The  denominator  consists  of  the  amount  of 
the  other  group  members’  total  speaking  time  that  A  looks  at  any  one  team  member  but 
does  not  talk.  The  formula  was  applied  to  each  actor  in  the  clip.  Koch  [43]  proposes 
further  adjustments  for  the  times  when  a  person  is  not  visible.  However,  for  this  sample 
our  adjustments  are  based  on  heuristics. 

The  results  of  the  analysis  are  presented  in  Table  7.  They  show  the  highest  VDR  for  RS 
(blue)  of  1 .04,  and  the  second-highest  for  GR  (grey)  of  0.92.  CS  (red)  and  LS  (green) 
both  had  substantially  lower  VDRs:  0.13  and  for  0.07  respectively. 

The  calculations  are  based  on  the  20  sec  video  and  the  results  are  likely  impacted  by  the 
staged  nature  of  the  scenario,  i.e.  “looking  away”  is  often  impacted  by  the  soldiers 
looking  at  their  notes,  i.e.  only  LS  and  RS  have  “looking  away”  time.  For  example,  the 
high  VDR  for  RS  is  likely  due  to  the  fact  that  he  was  looking  away  (at  notes),  but  also 
because  he  was  looking  away  while  talking  to  GR  for  a  brief  moment  -  a  sign  of 
inattention  or  “dominance”.  The  others  were  being  more  attentive  when  speaking,  i.e. 
their  ratios  of  “looking  while  talking/total  talking  time”  are  equal  and  therefore  1 .  The 
ratios  also  reflect  more  or  less  equal  dominance  between  GR  and  RS,  while  the  VDR  of 
CS  and  LS  are  much  lower.  It  should  also  be  noted  that  CS  (the  translator)  is  treated  the 
same  way  as  everyone  else,  even  though  his  assigned  role  is  one  of  a  facilitator,  that 
shouldn’t  (but  does)  impact  VDR  calculations.  For  example,  it  would  be  considered 
“more  polite”  if  soldiers  look  at  the  person  they  intend  to  talk  to,  instead  of  the  translator 
-  something  we  did  not  take  into  consideration  in  this  analysis. 

While  this  is  only  one  aspect  of  body  behaviors,  it  does  reflect  accorded  attention  (a 
positive  redress  behavior)  and  deference  or  lack  of  deference  (a  negative  redress 
behavior)  quite  powerfully  “redress”.  Over  time,  redress  effects  are  built  up.  In  this 
scenario,  RS  and  GR  are  clearly  the  more  powerful  individuals,  since  others  look  at  them 
when  they  are  talking,  while  CS  and  LS  are  lower  status  individuals.  The  notion  of 
dominance,  however  is  conveyed  by  “getting  away”  with  inattentiveness.  This  has  its 
parallel  in  Brown  and  Levinson’s  model  in  that  (all  other  things  being  equal)  more 
powerful  individuals  can  use  less  redress  to  achieve  balanced  face  threat  to  less  powerful 
individuals.  But  a  weakness  of  the  VDR  is  that,  unlike  Brown  and  Levinson,  it  does  not 
take  a  baseline  set  of  expectations  into  account.  Therefore,  while  it  is  true  that  the 
soldiers  generally  used  more  dominant  gaze  behaviors  in  this  scenario  (by  virtue  of  their 
looking  at  their  notes  or  each  other  rather  than  GR),  this  says  nothing  about  whether  an 
observer  felt  they  were  justified  in  doing  so  or  not.  As  illustrated  in  the  examples  in 
sections9.2  and  9.3  above,  at  least  given  our  formulations  of  the  beliefs  of  ourselves  as 
Observer,  this  was  not  the  case.  Instead  of  seeming  dominant,  the  soldiers  (particularly 
LS)  seemed  borderline  rude.  On  the  other  hand,  the  VDR  formula  provides  a  better  way 
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of  incorporating  all  the  gaze  and  speaking  time  data  from  all  the  individuals  in  an 
interaction  into  a  single  metric  than  does  our  politeness  calculations. 

It  should  be  noted  that  the  intervals  used  in  the  timeline  for  this  analysis  do  not  exactly 
match  those  in  Figures  18  and  19  used  in  the  politeness  analyses.  This  is  because  they 
were  done  at  different  times  by  different  individuals  who  had  different  interpretations  of 
various  action  intervals  in  the  video.  This  points  to  another  source  of  variability  in  the 
production  of  subtle  physiological  phenomena  such  as  these. 

10.  Summary  and  Conclusions 

The  results  of  this  extended  examination  of  the  applicability  of  our  politeness  model  and 
“Etiquette  Engine”  to  the  task  of  representing  body  behaviors  and  their  impact  on 
perceived  politeness  and  the  contributing  factors  which  politeness  mediates  (power, 
familiarity,  imposition,  character  and  the  value  of  redress)  shows  that  such  behaviors  can 
be  readily  incorporated  into  our  framework  and  body  behaviors  can  successfully  be 
represented  as  contributing  to  positive  or  negative  redress.  Furthermore,  the  combined 
effects  of  modeling  body  behaviors  as  redressive  behaviors,  using  scoring  techniques 
ported  from  our  initial  linguistic  approaches,  provided  reasonable  outcomes  ( in  that  they 
roughly  matched  our  intuitions  or  intentions  about  the  scenarios)  in  both  the  initial  textual 
“stories”  in  section  8.1  and  then  in  analyzing  the  “live”  training  interactions  in  section  9.2 
and  a  hypothetical  variant  on  it  in  9.3. 

We  cited  challenges  associated  with  continuous  body  behaviors  in  our  initial  list  of 
anticipated  challenges  (cf.  section  3)  but  this  may  turned  out  to  be  more  opportunity  than 
problem.  In  section  9.2.4  and  Figures  23  and  26,  we  attempted  a  technique  that  “spread” 
the  redressive  value  of  a  body  behavior  over  the  interval  of  that  behavior  and,  as  a  result, 
were  able  to  produce  moment-by-moment  charts  of  perceived  redress  which  track  an 
interaction.  We  know  this  approach  uses  many  simplifying  assumptions  (including 
avoiding  our  suspicions  that  the  impact  of  body  behaviors  grows  initially  and  then 
attenuates — as  illustrated  in  Figure  15)  and  it  ultimately  proved  less  useful  for  assessing 
the  “net”  imbalance  in  an  interaction  than  did  the  simple  difference  of  Redressive  value 
(counted  on  a  per  action  basis)  and  Face  Threat  Weight — as  we  had  used  previously. 

We  have  become  aware  that,  in  prior  work,  we  were  likely  subsuming  the  effect  of  body 
behaviors  in  our  scores  of  linguistic  behaviors.  Counting  both  of  them  together  likely 
overweights  the  redressive  value  present  in  an  interaction — and  will  tend  to  distort  the 
behavior  of  our  scales  and  their  presumed  “balance  point”  around  zero.  If  this  work  is 
carried  forward,  we  would  suggest  halving  the  value  of  both  linguistic  and  physical 
behaviors  relative  to  the  scales  and  values  assigned  in  the  examples  provided  here. 

Of  course,  substantial  challenges  remain.  Many  of  these  are  listed,  along  with  thoughts 
about  possible  solution  approaches,  in  section  8.2.  Chief  among  these  are  that  all  of  the 
scenarios  worked  in  this  document  involved  hand-crafted  recognition  of  both  the 
redressive  values  and  the  contributing  factors  pertinent  to  face  threat  weight.  Such 
capabilities,  in  all  their  subtlety,  cross  cultural  variation  and  context-dependency  (as  also 
cited  in  our  initial  challenge  list  in  section  3),  would  have  to  be  automated  for  the  full  use 
of  this  approach  in  a  simulation  or  gaming  environment.  Our  prior  work  has 
demonstrated  some  ability  to  accomplish  this  for  textual  entries,  but  doing  it  for  arbitrary 
body  behaviors  will  likely  be  more  challenging  still.  Acquiring  and  representing  this 
knowledge  is,  itself,  perhaps  the  greatest  practical  challenge — as  mentioned  in  8.2.9. 


SIFT-SSIM-Report-v08-fin 


70  of  74 


12/1/2014  8:22:00  PM 


s 


Smart  Information  Flow  Technologies 


But  we  have  demonstrated  that  our  etiquette -based  approach  holds  promise  and  can  be 
extended  to  capture  specific  types  of  body  behaviors  with  redressive  value  along  with 
their  effects  on  overall  perceived  politeness.  Future  research  and  development  may 
explore  more  specific  applications  and  seek  to  solve  problems  that  we  can  only  note  and 
suggest  approaches  to  addressing. 
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Executive  Summary 

The  42-month  DARPA  program,  Strategic  Social  Interaction  Modules  (SSIM)  has  come  to  an 
end,  and  the  purpose  of  this  document  is  to  describe  the  contributions  of  MacroCognition  LLC  to 
the  effort  under  Raytheon  BBN  Technologies  Prime  Contract  No:  W91 1-12-0002  (Purchase 
Order:  9500011008).  The  intent  of  the  overall  program,  nicknamed  the  Good  Strangers  program 
was  to  enable  warfighters  to  learn  the  social  skills  necessary  to  have  successful  social  encounters 
with  civilians  while  stationed  in  foreign  countries.  The  objective  of  Raytheon  BBN  Technologies 
was  to  design  and  implement  innovative  tools  for  training  human  interaction  dynamics.  This 
training  will  be  implemented  in  an  immersive,  interactive,  simulation  environment  that  provides 
the  trainee  with  the  opportunity  to  develop,  exercise,  and  adapt  social  interaction  skills,  while 
executing  specific  military  tasks  and  engaging  virtual  humans  in  unfamiliar  cultural,  linguistic, 
geopolitical,  and  geographic  settings.  MacroCognition  LLC’s  goal  was  to  assist  in  the  design  and 
implementation  of  training  tools  through  two  major  efforts  -  1)  the  GS  diagnostic  tool  (Gap 
Study),  and  2)  Gary  Klein  consultation. 

Using  Cognitive  Task  Analysis  (CTA)  interview  data  collected  in  Task  Area  lB-Phase  I, 
MacroCognition  LLC  constructed  a  summative  study  that  sought  to  identify  the  cognitive, 
behavioral  and  perceptual  gaps  between  the  Good  Stranger  (GS)  skills  needed  and  those 
possessed  by  the  military  personnel.  Researchers  and  practitioners  are  often  tempted  to  transform 
expert  knowledge  and  skills  into  training  requirements.  However,  this  is  a  mistake  because 
trainees  are  not  all  alike.  Some  trainees  already  possess  these  advanced  skills,  while  others  may 
have  deficits.  Therefore,  our  goal  was  to  investigate  the  training  requirements  for  GS  skills 
(identified  by  the  TA1B  team  and  our  CTA  research;  see  Appendix  A),  and  examine  a  general 
military  population  to  assess  which  of  the  GS  skills  identified  were  relevant  as  training 
objectives.  We  constructed  the  GS  diagnostic  tool  obtain  a  rough  estimate  of  the  number  of 
soldiers  that  already  identify  as  GSs,  and  to  measure  the  existing  gap  between  the  current  skill 
set  of  soldiers  and  the  skill  set  needed  to  be  GSs  (see  appendix  B). 

An  exploratory  study  was  performed  at  Ft.  Benning  using  138  Non  Commissioned  Officers 
(NCOs)  who  individually  completed  an  extensive  diagnostic  assessment  containing  scenarios 
and  forced  choice  responses  (e.g.  ranking  alternatives,  multiple  choice,  Likert  Scales;  see 
Appendix  C).  Because  of  errors  and  incomplete  responses,  only  78  NCOs  generated  usable  data 
records.  The  study  employed  three  distinct  sections  (Frames,  Learning  Objectives,  and 
Competency)  to  address  the  following  training  questions. 

1.  Do  the  NCOs  self-identify  as  GSs? 

2.  When  given  forced  choice  questions  with  other  high  level  goals  (e.g.  security,  mission 
accomplishment),  are  their  choices  consistent  with  GS  behaviors  (e.g.  de-escalation, 
rapport  building,  voluntary  compliance)? 

3.  When  queried  about  de-escalating  situations  and  handling  civilian  challenges,  do  they 
provide  evidence  of  the  GS  skills? 

Theoretical  Contributions 

Using  the  diagnostic  assessment,  we  found  that  GS  frame  is  significantly  associated  with 
preference  for  GS  skills  (e.g.  de-escalation,  building  rapport)  and  negatively  associated  with 
excessive  security.  Further,  authoritarian  frame  preference  was  significantly  associated  with 


30  March  2015 


MacroCognition  llc 


decreased  prioritization  and  understanding  for  certain  trust  building  tactics  (see  Appendix  D). 
These  results  further  confirm  the  link  between  the  GS  frame  and  specific  trust  building  tactics 
(G.  Klein,  H.  Klein,  Lande,  Borders  &  Whitacre,  2015). 

Training  Implications 

Our  findings  indicate  that  most  of  the  NCOs  self-reported  as  GSs.  More  specifically,  they  highly 
prioritized  options  manifesting  the  GS  framework  in  comparison  to  other  cognitive  frames  (e.g. 
mission,  rules  &  procedures,  authoritarian).  Although  most  preferred  the  GS  frame,  far  fewer 
soldiers  showed  GS  proclivities  or  competency  for  the  GS  skills  such  as  rapport  building  and 
perspective  taking.  Perhaps  most  striking,  soldiers  least  preferred  and  were  least  confident 
performing  rapport  building  actions,  such  as  showing  empathy  and  searching  for  common 
ground.  For  more  training  suggestions  see  Appendix  E. 

General  Consulting 

An  ongoing  collaborative  effort  between  MacroCognition  LLC  and  Raytheon  BBN  Technologies 
included,  Gary  Klein  providing  general  consulting  on  the  design  and  implementation  of 
simulation  tools  for  training  cultural  interaction  skills.  He  provided  his  expertise  in  decision 
making,  common  ground,  cultural  cognition,  training,  sensemaking,  and  expertise  to  advise  what 
training  objectives  need  to  be  emphasized  and  what  can  be  minimized  -  distinguishing  the  low- 
hanging  fruit  (low  cost,  low  effort,  high  impact)  from  high  hanging  fruit  (technological 
challenges  with  little  payoff). 

More  specifically,  Gary  Klein  provided  on-call  consultation  on  the  Task  Area  II  effort.  This 
took  the  form  of  telephone  calls  and  participation  in  the  TA2  teleconferences,  particularly  in  the 
early  stages  of  the  program,  multiple  visits  to  BBN  headquarters,  and  a  site  visit  to  UC/Santa 
Cruz.  The  purpose  of  these  activities  was  to  share  the  results  of  the  CTA  study  from  Task  Area 
1A  with  BBN  and  its  partners  in  order  to  improve  the  design  of  the  simulation. 

Summary 

MacroCognition  has  made  a  variety  of  contributions  to  the  BBN  effort: 

•  Creating  a  sensemaking  model  of  a  GS 

•  Providing  consultation  on  the  design  and  implementation  of  the  simulation  tools 

•  Constructing  a  usable  diagnostic  tool 

o  Assessing  the  GS  potential  of  NCOs,  to  become  GSs 

o  Provide  relevant  training  suggestions  for  researchers,  trainers  and  practitioners 
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Good  Stranger  KSAs 

Gary  Klein  &  Helen  Altman  Klein 

The  purpose  of  this  memo  is  to  list  the  features  that  differentiate  Good  Strangers 
(primarily  in  the  military)  from  others.  These  features  are  the  basis  for  categorizing 
individuals  as  GSs.  They  also  suggest  themes  for  building  scenarios  -  types  of  choices 
that  will  help  shift  people  to  a  GS  frame. 

1.  Preferring  voluntary  to  coercive  compliance  from  civilians. 

2.  Looking  for  chances  to  interact  with  civilians  and  build  trust. 

a.  Seeks  to  gain  at  least  basic  language  skill. 

b.  Eager  to  adopt  local  customs  and  standards  of  polite  behavior. 

3.  Being  irritated  with  Bad  Strangers  -  colleagues  who  can  get  them  killed. 

4.  Appreciating  role  models  who  are  GSs. 

a.  Speaking  softly  and  calmly. 

b.  Not  getting  agitated. 

c.  Not  generating  fear  or  antagonism. 

5.  Refusing  to  take  provocations  personally. 

a.  Able  to  laugh  at  self;  doesn’t  take  self  too  seriously. 

6.  Ready  to  resolve  disagreements  rather  than  avoiding  them. 

7.  Curious  about  the  reasons  for  odd  behavior. 

a.  Especially  when  dealing  with  foreign  cultures. 

b.  Able  and  eager  to  take  another  perspective. 

c.  Assuming  there  is  probably  a  good  reason  for  the  behavior. 

8.  Aware  of  long-term  consequences  of  (positive)(negative)  encounters. 

9.  Appreciating  the  tricks  of  the  trade  used  to  gain  voluntary  compliance. 

a.  Verbal  formulas 

b.  Win/win  ways  to  frame  disputes. 

10.  Desiring  a  reputation  for  being  fair  and  trustworthy. 

11.  Careful  not  to  project  arrogance  or  domination  or  intimidation. 

12.  Able  to  defuse  a  conflict.  Prepared  to  make  himself/herself  vulnerable. 

13.  Believes  in  “policing  our  own.”  Speaks  out  when  colleagues  are  out  of  line. 
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The  SSIM  Gap  Study 


U.  S.  military  personnel  are  deployed  in  more  than  150  countries  around  the  world. 
This  means  that  about  160,000  active-duty  personnel  serve  outside  the  United 
States  and  its  territories i. 

The  Gap  Study  is  sponsored  by  the  Defense  Advanced  Research  Projects  Agency 
(DARPA).  It  is  designed  to  collect  information  that  can  provide  better  support  to 
U.S.  military  personnel  deployed  abroad.  The  training  that  will  be  developed  from 
this  study  will  only  be  as  good  as  the  information  you  share  with  us  today.  We  will 
ask  you  to  make  decisions  about  real  incidents  that  U.S.  military  personnel 
experienced  during  foreign  deployments.  We  have  selected  these  incidents  from 
hundreds  of  pages  of  interview  transcripts  with  U.S.  soldiers  about  their 
experiences  overseas  and  from  other  print  and  audio  records. 

The  GAP  material  has  three  sections.  In  Section  1,  you  will  review  10  brief 
scenarios  and  decide  on  the  course  of  action  that  you  would  choose  for  each.  In 
Section  2,  you  will  review  5  complex  scenarios  -  each  presenting  different 
challenges,  suggesting  different  courses  of  action,  and  offering  different  choices. 
You  will  be  asked  about  your  concerns,  priorities,  and  choices.  In  Section  3,  you 
will  be  asked  how  you  would  accomplish  the  goals  for  scenarios  described  in 
Section  2. 

1  "Total  Military  Personnel  and  Dependent  End  Strength  By  Service,  Regional  Area,  and  Country".  Defense 
Manpower  Data  Center.  March  31,  2014. 


Before  beginning  this  study,  we  need  to  review  the  Consent  Form  that  details 
your  rights  as  a  participant  and  our  obligations  as  researchers.  Please  wait  for 

instructions  before  continuing. 
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I.  Brief  Scenarios 


I.  Brief  Scenarios 


Read  each  of  the  following  10  scenarios  and  then  rank  the  four  actions:  1  =  the  action  that 
you  believe  is  the  best  choice,  4  =  the  action  you  believe  is  the  worst  choice.  For  each  scenario, 

use  each  ranking  (1,  2,  3,  &  4)  only  once. 

1)  Illegal  Weapon.  You  are  at  a  checkpoint  searching  for  weapons.  An  Iraqi  man  enters  the 
checkpoint.  The  man’s  background  check  comes  out  clean  but  you  see  what  looks  like  a  pistol 
under  a  blanket  in  the  backseat.  You  know  that  pistols  are  not  allowed  in  vehicles.  Rank  these 
actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Investigate  the  backseat  to  confirm  the  presence  of  a  weapon.  Follow  the  procedure 
for  confiscating  and  securing  an  illegal  weapon. 

b.  Direct  the  man  to  drive  his  vehicle  out  of  the  way  to  keep  the  checkpoint  moving. 
Once  off  to  the  side,  explain  your  responsibility  for  public  safety. 

c.  Ask  the  man  why  he  is  smuggling  the  pistol  though  the  gate.  While  you  can’t  arrest 
him,  detain  him  for  a  while  to  show  that  you  are  in  control. 

d.  Ask  the  man  why  he  needs  a  pistol  and  what  threats  worry  him.  Explain  that  your 
mission  is  maintaining  safety. 

2)  Guarding  the  Money.  You  are  leading  a  small  team  escorting  a  shipment  of  money  to  a  local 
Iraqi  bank.  The  bank  has  not  been  able  to  pay  pensioners  for  days.  This  has  upset  many  local 
people.  When  you  arrive,  you  see  elderly  people  gathering  around  the  bank  and  demanding  their 
money.  The  bank  manager  tells  you  that  they  will  not  be  able  to  open  the  bank  until  the  next  day. 
He  must  first  take  inventory  of  the  money.  As  you  finish  the  escort,  you  notice  a  growing  crowd 
of  elderly  civilians  moving  towards  the  bank  doors.  Rank  these  actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Establish  a  perimeter  around  the  bank  to  prevent  anyone  from  entering.  Have  your 
soldiers  stand  with  their  rifles  visible  to  establish  your  authority 

b.  Have  the  bank  manager  address  the  local  people.  He  should  apologize  for  the  delay 
and  explain  that  the  bank  will  open  the  next  day. 

c.  Complete  your  escort  mission  and  inform  the  bank  manager  that  he  is  in  charge  of 
handling  the  situation  as  you  are  leaving. 

d.  Carefully  follow  all  rules  regarding  interactions  with  local  business  owners  and 
citizens. 
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I.  Brief  Scenarios 


Read  each  of  the  following  10  scenarios  and  then  rank  the  four  actions:  1  =  the  action  that 
you  believe  is  the  best  choice ,  4  =  the  action  you  believe  is  the  worst  choice.  For  each  scenario, 

use  each  ranking  (1,  2,  3,  &  4)  only  once. 

3)  Local  Laborers.  A  number  of  local  people  work  on  the  base  to  support  maintenance 
operations.  Each  day,  a  few  of  them  arrive  for  work  after  the  beginning  of  their  shift.  This  makes 
it  difficult  to  complete  the  assigned  maintenance  tasks  and  puts  pressure  on  the  other  workers. 
Rank  these  actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Fire  the  worst  offender  and  announce  that  you’ll  no  longer  tolerate  late  arrivals. 
Other  workers  will  understand  that  promptness  is  important. 

b.  Review  the  work  schedule  to  see  if  there  might  be  a  way  to  stagger  shifts  to  get  the 
work  done  and  still  accommodate  different  arrival  times. 

c.  Talk  with  the  men  who  habitually  arrive  late  to  better  understand  why.  If  possible, 
address  the  reason  for  their  problem(s). 

d.  Review  the  job  description  to  see  if  the  time  of  arrival  is  clearly  stated  and  if  any 
penalties  for  late  arrival  are  specified.  Follow  the  guidelines. 

4)  Suspicious  Behavior.  You  are  assisting  at  a  checkpoint  near  a  neighborhood  known  to  harbor 
insurgents.  Protocol  requires  that  you  thoroughly  check  every  car  that  enters  and  exits.  A  local 
resident  you  recognize  drives  up  to  the  checkpoint.  The  man  is  usually  pleasant  and  co-operative. 
Today,  however,  he  says  he  needs  to  rush  and  asks  that  the  check  be  hurried.  Rank  these  actions 
from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Examine  the  car  and  ask  the  questions  that  are  on  your  checklist.  Although  he  is  in 
a  rush,  do  not  express  favoritism  towards  the  man. 

b.  Ask  why  he  is  in  a  hurry  so  you  can  to  identify  potential  village  security  problems. 
Carry  out  the  search  as  quickly  as  possible. 

c.  The  man’s  urgency  is  suspicious.  If  he  doesn’t  respond  to  your  questions,  remove 
him  from  his  vehicle  forcefully  if  necessary.  Search  him  and  his  car. 

d.  Expedite  the  check  as  long  as  the  man  is  compliant.  Watch  for  changes  in  his 
behavior  that  might  signal  potential  threats.  Keep  alert  to  oncoming  vehicles. 
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I.  Brief  Scenarios 


Read  each  of  the  following  10  scenarios  and  then  rank  the  four  actions:  1  =  the  action  that 
you  believe  is  the  best  choice ,  4  =  the  action  you  believe  is  the  worst  choice.  For  each  scenario, 

use  each  ranking  (1,  2,  3,  &  4)  only  once. 

5)  Disaster  Relief.  Your  squad  is  providing  security  assistance  during  a  disaster  relief  operation 
in  Chile.  As  you  finish  a  regularly  scheduled  neighborhood  walk-through,  a  team  member  spots  a 
stranger  running  towards  a  nearby  house.  You  are  familiar  with  the  family  that  lives  in  the  house. 
You  approach  the  house  and  ask  the  homeowner  if  he  has  seen  strangers  in  the  area  or  heard 
anyone  trying  to  enter  his  house?  The  man  says  “No”  and  starts  to  close  his  door.  Rank  these 
actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Allow  him  to  save  face.  Suggest  that  a  stranger  may  have  entered  through  a  back 
window  without  his  knowledge.  Search  the  home  respectfully. 

b.  Insist  that  you  search  his  home.  Check  the  house  carefully  following  the  rules  and 
procedures  specified  for  this  situation. 

c.  State  that  you  must  search  his  home.  Enter  the  home  without  consent  if  necessary. 
Take  any  steps  needed  to  subdue  resistance  and  ensure  security. 

d.  Your  mission  is  to  reduce  potential  threat.  Before  searching  the  house,  have  the 
man  move  his  family  to  a  safe  location. 

6)  Angry  Teens.  You  and  your  men  are  inspecting  an  Iraqi  radio  intercept  site  you’ve  found. 
You’ve  been  in  this  town  for  some  time  without  any  hostilities.  As  you  carry  out  the  inspection, 
a  group  of  teenage  males  approach  the  site.  You  suspect  that  they  are  looking  for  bags  of  food 
they  stashed  at  the  site  and  that  you  are  in  their  way.  They  don’t  appear  to  have  weapons,  but 
they  look  angry.  Rank  these  actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Greet  the  teenagers  without  drawing  your  weapons  because  your  team  is  alert  to 
potential  danger.  Ask  the  teens  if  you  can  help  them. 

b.  Complete  the  inspection  as  required  keeping  a  careful  eye  on  the  teenagers.  Follow 
rules  related  to  any  potentially  hostile  civilian  actions. 

c.  Have  most  of  your  team  complete  the  inspection.  You  and  several  others  should 
engage  the  teens  to  identify  possible  threats. 

d.  Intimidate  the  teenagers  by  ordering  your  men  to  get  into  a  low  ready  position. 

Direct  the  teenagers  to  follow  your  commands  so  you  can  search  them  for  weapons. 
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I.  Brief  Scenarios 


Read  each  of  the  following  10  scenarios  and  then  rank  the  four  actions:  1  =  the  action  that 
you  believe  is  the  best  choice ,  4  =  the  action  you  believe  is  the  worst  choice.  For  each  scenario, 

use  each  ranking  (1,  2,  3,  &  4)  only  once. 

7)  Security  vs.  Convenience.  You  are  stationed  at  a  U.S.  base  that  also  houses  a  contingent  of 
Afghani  soldiers.  A  gate  near  the  Afghani  quarter  is  very  close  to  their  latrine  but  it  must  be  kept 
locked  at  night  for  security  reasons.  The  Afghanis  have  been  told  that  the  gate  must  always  be 
locked.  The  gate,  however,  is  often  left  unlocked.  Rank  these  actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Review  the  reasons  for  your  concern  with  the  Afghani  soldiers.  Try  to  find  a 
solution  with  them  so  that  you  can  avoid  locking  the  gate  indefinitely. 

b.  Tell  the  Afghanis  the  short-term  and  long-term  consequences  you  will  have  to 
impose  if  the  gate  is  found  opened  again.  Keep  your  word  to  the  letter. 

c.  You  can’t  take  security  lightly.  Lock  the  gate  the  next  time  it  is  found  opened.  Tell 
them  you  will  open  it  as  soon  as  they  come  up  with  a  viable  solution. 

d.  Your  concern  is  security,  not  their  convenience.  Next  time  it  is  left  unlocked,  order 
that  it  is  permanently  closed. 

8)  Mine  detection  in  South  Sudan.  As  part  of  a  U.N.  peacekeeping  mission,  you  have  orders  to 
support  a  land  mine  detection  team  as  they  search  a  wooded  area.  Because  this  can  cause 
dangerous  explosions,  families  living  nearby  were  ordered  to  leave  their  homes  and  land  for 
several  days.  They  were  promised  monetary  compensation  for  their  trouble.  When  you  arrive, 
you  find  that  one  family  has  not  started  to  prepare.  It  is  important  that  the  family  leaves  so  that 
the  work  can  begin.  Rank  these  actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  Order  the  owner  to  immediately  gather  his  necessities  because  he  is  delaying  the 
detection  team’s  work.  Tell  the  team  they  can  get  to  work. 

b.  Reassure  the  owner  that  his  valuables  will  be  safe  while  he  is  gone.  This  may  delay 
the  work  a  bit  but  will  create  a  better  feeling. 

c.  The  owner  had  time  to  gather  his  possessions  before  you  arrived.  If  he  is 
noncompliant,  remove  him  and  his  family  forcefully. 

d.  Assign  your  squad  members  to  help  the  owner  gather  his  valuables  if  this  will 
allow  you  to  get  work  started  sooner. 
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I.  Brief  Scenarios 


Read  each  of  the  following  10  scenarios  and  then  rank  the  four  actions:  1  =  the  action  that 
you  believe  is  the  best  choice ,  4  =  the  action  you  believe  is  the  worst  choice.  For  each  scenario, 

use  each  ranking  (1,  2,  3,  &  4)  only  once. 

9)  A  Negligent  Local  Contractor.  A  local  contractor  was  hired  to  remove  trash  from  a  school 
construction  site.  He  was  told  that  the  work  had  to  be  completed  in  two  weeks  so  that  the  school 
could  open  on  schedule.  Three  days  before  the  deadline  you  notice  that  he  has  not  yet  started 
work.  Rank  these  actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  If  you  are  not  contractually  required  to  pay  the  contractor,  dismiss  him  without  pay. 

b.  Tell  the  contractor  that  school  must  open  on  time.  Tell  him  you  must  replace  him 
and  that  you  will  work  closely  with  him  if  he  is  employed  at  the  base  in  the  future. 

c.  Fire  the  contractor  and  let  the  community  know  the  delay  was  the  contractor’s  fault. 
Local  people  will  understand  that  they  need  to  honor  commitments  to  the  U.S  Army. 

d.  Replace  the  contractor  immediately  with  one  who  is  available  and  who  has  a 
reputation  for  good  work  even  under  pressure. 

10)  Neighborhood  patrol.  Your  squad  is  patrolling  a  neighborhood  to  provide  security.  These 
patrols  are  usually  uneventful  as  the  neighborhood  has  been  peaceful  for  quite  a  while.  A  team 
member  spots  a  young  child  dropping  what  looks  to  be  a  grenade  from  a  second-story  window  of 
a  home.  He  yells,  “Run!  Run!”  Fortunately,  there  was  no  explosion  and  no  panic.  You  see  that  it 
was  actually  a  stone.  Rank  these  actions  from  best  (1)  to  worst  (4). 


Rank 

Actions 

a.  This  prank  could  have  lead  to  serious  consequences.  Enter  the  home  and  tell  the 
parents  that  it  is  their  responsibility  to  prevent  this  from  happening  again. 

b.  Before  you  continue  your  patrol,  record  the  details  of  this  incident  in  your  field 
journal.  It’s  your  job  to  keep  a  log  of  all  unusual  events. 

c.  Though  unsettling,  this  incident  should  not  interfere  with  your  checking  the 
remaining  area.  The  incident  may  have  been  a  distraction. 

d.  Knock  at  the  door  and  tell  the  family  what  you’ve  observed.  Take  the  time  to 
explain  your  responsibility  for  neighborhood  safety. 
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Please  wait  for  directions  to  begin  the  next  section. 
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II.  Complex  Scenarios:  #1  Broadsided 


II.  Complex  Scenarios 


Scenario  #1:  Broadsided 


Background 

You  are  a  staff  sergeant  in  a  brigade  motor  pool.  You  have  heard  rumors  that  the  Iraqis  are 
intentionally  hitting  U.S.  vehicles  to  collect  compensation  from  the  U.S.  Government.  You 
recently  received  a  memo  alerting  drivers  to  be  especially  careful  and  to  stay  on  lookout  for 
accidents  that  look  suspicious. 

While  working  dispatch,  you  receive  a  radio  call  from  a  four-vehicle  convoy.  One  of  their  trucks 
has  had  an  accident  with  a  civilian  vehicle.  There  are  no  serious  injuries  and  the  truck  does  not 
need  a  tow,  but  the  soldier  reporting  the  accident  sounds  very  tense  and  says  you  should  get  out 
there  soon. 

When  you  arrive  at  the  accident  site,  you  see  the  U.S.  truck  in  contact  with  an  Iraqi  vehicle’s 
damaged  hood.  Soldiers  have  set  up  security  around  the  vehicles.  You  see  SGT  Lawrence,  the 
motor  transport  section  leader,  in  a  shouting  match  with  the  Iraqi  driver  who  had  collided  with 
the  truck.  They’re  arguing  about  whose  fault  it  was  and  who  has  to  pay  for  damages.  SGT 
Lawrence  is  accusing  the  Iraqi  of  causing  the  accident  intentionally. 

The  damaged  vehicle  is  a  taxi  owned  and  driven  by  a  local  man  named  Walid.  He  occasionally 
comes  to  the  base  to  provide  transportation  for  base  personnel.  When  you  arrive,  SGT  Lawrence, 
who  is  not  on  duty,  is  out  of  control.  He  is  shouting  at  Walid  using  very  offensive  language. 
Walid  is  responding  in  Arabic. 


If  you  were  in  this  situation,  circle  (1-5)  how  important  it 

Very  Important  Not  Important 

would  be  to  know  what  Walid  is  thinking? 

1  2  3  4  5 

A  dozen  Iraqi  soldiers  from  a  nearby  checkpoint  have  come  over  to  watch.  They  are  cheering  for 
Walid  and  becoming  increasingly  angry  at  SGT  Lawrence’s  behavior.  Some  of  the  American 
soldiers  are  shouting  back.  Iraqi  soldiers  are  fanning  out  around  the  scene  and  some  have 
unslung  their  weapons.  SGT  Lawrence  yells  at  the  American  soldiers  to  stand  ready  and  be 
prepared  to  use  force. 
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II.  Complex  Scenarios:  #1  Broadsided 


Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Order  SGT  Lawrence  to  return  to  the  base  to  reduce  tensions. 

b.  Listen  attentively  to  Walid  so  those  watching  will  trust  you  to  find  a  fair  resolution. 

c.  Convince  the  Iraqis  that  the  problem  will  be  resolved  faster  if  they  return  to  their 
checkpoint. 

cl.  Reduce  immediate  risk  by  ordering  your  men  to  take  cover  behind  their  vehicles 
while  you  talk  to  the  crowd. 

Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Get  SGT  Lawrence  to  return  to  the  base. 

1  2  3 - 4  5 

Increase  the  trust  Walid  and  the  others  have  for  you. 

1  2  3 - 4  5 

Get  the  Iraqi  soldiers  to  peacefully  return  to  their 
checkpoint. 

1  2  3  4  5 

Keep  your  men  safe  by  having  them  move  behind  the 
vehicles. 

1  2  3  4  5 

Scenario  Continues 

You  are  able  to  convince  SGT  Lawrence  to  return  to  the  base  and  to  convince  your  men  to  take 
positions  behind  the  vehicles.  Meanwhile,  an  Iraqi  police  officer  arrives.  You  both  listen  as 
Walid  describes  what  had  happened  and  how  SGT  Lawrence  started  shouting.  Looking  at  the 
two  vehicles,  it  is  your  impression  that  this  was  a  genuine  accident. 

The  Iraqi  police  officer  demands  the  U.S.  truck  driver’s  Identification  Document.  He  says  this  is 
required  under  Iraqi  law.  You  explain  that  U.S.  soldiers  are  not  allowed  to  surrender  their  IDs. 
The  police  officer  continues  his  request  and  the  Iraqi  soldiers  are  getting  more  agitated.  They 
continue  shouting  and  mocking  SGT  Lawrence  and  the  other  Americans.  As  the  dispute 
continues,  you  notice  traffic  building  up  behind  the  accident.  You  fear  that  Iraqi  citizens  will 
begin  leaving  their  vehicles  to  see  what  is  going  on. 
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II.  Complex  Scenarios:  #1  Broadsided 


Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Take  pictures  of  the  vehicles  so  that  Walid  and  the  onlookers  see  that  you  are 
objectively  documenting  the  damage  to  the  car. 

b.  Ask  the  Iraqi  police  officer  to  help  you  disperse  the  crowd  and  to  help  you  prevent  a 
traffic  build-up. 

c.  Direct  your  attention  to  monitoring  and  controlling  the  crowd  of  Iraqi  soldiers 
because  they  are  the  immediate  threat. 

cl.  Explain  to  the  Iraqi  police  officer  that  your  soldiers  are  not  allowed  to  hand  over  their 
ID  cards.  Assure  him  that  an  acceptable  document  will  be  available  tomorrow. 

Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Convince  Walid  and  the  onlookers  that  you  want  to  resolve 
the  problem  fairly. 

1  2  3  4  5 

Get  the  Iraqi  police  officer  to  help  disperse  the  crowd. 

1  2  3  4  5 

Monitor  and  control  the  crowd  of  Iraqi  soldiers. 

1  2  3  4  5 

Satisfy  the  police  officer  that  he  will  have  an  acceptable 
document  for  the  soldier  the  next  day. 

1  2  3  4  5 

End  Scenario  #1:  Broadsided 


II.  Complex  Scenarios:  #2  Shots  into  the  Crowd 


Scenario  #2:  Shots  into  the  Crowd 


Background 

You  are  a  rifle  platoon  commander.  You’ve  just  been  assigned  to  a  Shiite  town  in  Iraq.  Earlier 
this  year  Special  Forces  had  arrested  the  town’s  Police  Chief  for  taking  weapons  from  the  police 
armory  and  sending  them  to  insurgent  groups.  Before  he  was  captured,  the  insurgency  group  had 
terrorized  the  local  community.  You  have  set  up  your  base  of  operations  at  a  TV  station  just 
outside  of  town.  The  first  thing  you  do  is  to  arrange  a  meeting  with  the  town’s  council  at  the 
community  center.  You  arrive  with  a  security  force  that  waits  outside  the  meeting  place. 


If  you  were  in  this  situation,  circle  (1-5)  how  important  it 
would  be  to  know  how  the  council  members  view 
Americans? 


Very  Important  Not  Important 
1 - 2 - 3 - 4 - 5 


At  the  meeting,  you  plan  to  explain  how  your  team  will  provide  security.  You  also  want  to  learn 
who  are  the  important  citizens  and  who  you  can  trust.  You  would  like  to  avoid 
misunderstandings  as  much  as  possible.  You  include  your  translator  to  help  communication  at 
the  meeting.  What  are  the  most  important  objectives  that  you  want  to  convey  to  the  council? 

Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Convince  the  local  council  that  you  will  be  able  to  improve  security  if  they  provide 
information  about  potential  threats. 

b.  Stay  alert  for  individual  council  members  who  seem  likely  to  become  threats  in  the 
future  and  those  who  seem  likely  to  become  allies. 

c.  During  the  meeting,  focus  your  attention  on  their  concerns  and  their  hopes. 

d.  Acknowledge  the  town’s  problems  in  the  past  with  insurgent  groups  and  indicate 
your  intent  to  improve  security  and  minimize  threats. 
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II.  Complex  Scenarios:  #2  Shots  into  the  Crowd 


Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Increase  the  council’s  trust  and  their  co-operation  in  security 
matters. 

1  2  3  4  5 

Identify  potential  threats  among  the  council  members  and 
cultivate  potential  allies. 

1  2  3  4  5 

Encourage  the  council  to  convey  their  wants  and  needs  so 
that  you  can  incorporate  them  into  your  planning. 

1  2  3  4  5 

Keep  tensions  low  by  explaining  your  future  intensions  to 
increase  security. 

1  2  3  4  5 

Scenario  Continues: 

The  meeting  lasts  several  hours.  You  try  to  answer  their  questions  as  honestly  and  directly  as 
possible.  There  are  no  major  problems  but  several  council  members  appear  to  be  uneasy  and 
seem  skeptical  about  you  and  your  motivations. 

The  meeting  has  ended  satisfactorily  and  you  walk  away  from  the  table  with  your  translator.  As 
you  leave  the  community  center  you  first  notice  that  the  villagers  have  gathered  to  greet  you  and 
you  then  hear  gunfire.  You  immediately  assume  the  worst — an  attack  on  your  security  force 
waiting  outside  the  building.  Suddenly  one  of  your  men  yells,  “Sir,  it  was  me,  I  fired  in  error!” 
You  see  that  a  10-year-old  and  two  teenagers  have  been  shot.  Understandably,  the  villagers  are 
reacting  with  agitation  and  screaming.  You  need  to  act  quickly  to  avert  harm  and  identify  your 
best  course  of  action. 

Your  priorities:  How  important  is  each  of  these  actions?  Review  cdl  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Action 

a.  Direct  your  platoon  to  an  exterior  defense  posture  in  case  there  is  retaliation. 

b.  Inform  the  villagers  that  your  medics  will  provide  first  aid,  and  assess  the  injuries  to 
determine  what  additional  treatment  will  be  needed. 

c.  Order  your  men  to  lower  their  weapons  to  acknowledge  your  responsibility  and  as  a 
sign  of  respect  for  the  injured  children. 

cl.  Ask  the  crowd  to  clear  the  immediate  area  to  make  it  easier  for  your  medics  to 
initiate  diagnoses  and  medical  care. 
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II.  Complex  Scenarios:  #2  Shots  into  the  Crowd 


Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Take  the  necessary  security  precautions  in  case  of  an  attack. 

1— 

2 

-3— 

4 

—5 

Provide  assurance  to  the  villagers. 

1— 

2 

-3— 

4 

—5 

Reduce  hostility  and  cultivate  a  sense  of  common  concern. 

1— 

2 

-3— 

4 

—5 

Persuade  the  crowd  to  clear  the  area. 

1— 

2 

-3— 

4 

—5 

You  immediately  announce  your  responsibility  and  order  your  medics  to  provide  medical 
attention.  This  quiets  the  gathered  crowd.  The  teenagers’  wounds  seem  to  be  minor.  They  are 
treated  on  site  by  your  medics.  However,  the  10-year-old  is  injured  pretty  badly.  He  and  his 
parents  are  taken  to  a  larger  medical  facility  several  miles  away  for  further  evaluation  and 
treatment. 


Scenario  Continues: 

The  next  day  you  are  faced  with  a  difficult  decision.  Before  the  accident  happened,  you  had 
scheduled  another  meeting  with  the  town  leaders.  You  need  to  maintain  a  constructive 
relationship  with  the  town  in  order  to  accomplish  your  mission  but  you  also  want  to  limit  risk. 
How  should  you  approach  this  follow-up  meeting? 

Your  priorities:  How  important  is  each  of  these  actions?  Review  cdl  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Begin  the  meeting  by  reporting  the  medical  condition  of  the  child.  Indicate  that  he 
will  continue  to  receive  needed  medical  care. 

b.  To  cultivate  good  will  and  a  shared  concern,  go  to  the  meeting  with  limited  security 
and  armor.  Walk  into  the  meeting  room  without  your  Kevlar  and  helmet. 

c.  Because  of  the  civilian  injuries  your  team  caused,  go  to  the  meeting  with  increased 
security  (more  troops)  in  case  of  hostility  or  retaliation. 

cl.  Explain  that  their  immediate  cooperation  in  clearing  the  way  for  the  emergency 
vehicles  helped  get  the  child  to  medical  care.  This  begins  a  cooperative  relationship. 
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II.  Complex  Scenarios:  #2  Shots  into  the  Crowd 


Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Reduce  the  councils’  anger  by  reporting  on  the  child’s 
condition. 

1  2  3  4  5 

Reduce  hostilities  by  going  to  the  meeting  with  limited 
security  and  minimal  armor. 

1  2  3  4  5 

Maintaining  security  by  taking  protective  safety  measures  to 
reduce  the  chances  of  an  attack. 

1  2  3  4  5 

Increase  community  cooperation  in  the  future  by  pointing 
out  its  role  after  the  shooting. 

1  2  3  4  5 

End  Scenario  #2:  Shots  into  the  Crowd 


II.  Complex  Scenarios:  #3  Guard  Tower 


Scenario  #3:  Guard  Tower 


Background 

You  are  part  of  a  multinational  team  assigned  to  maintain  security  in  a  remote  area  of  Iraq.  Your 
job  is  to  oversee  the  guard  tower  forces  that  maintain  security  operations  around  the  perimeter 
and  entrances  to  the  base.  You  have  about  14  skilled  U.S.  troops  as  well  as  about  40  Iraqi 
nationals.  With  the  exception  of  your  assigned  translators,  few  of  the  U.S.  personnel  know  more 
than  basic  Arabic.  The  Iraqis  have  had  no  English  language  instructions  but  some  of  them  are 
picking  up  language  by  listening  to  the  banter  of  their  American  counterparts. 

While  security  is  your  most  important  duty,  your  base  commander  has  also  emphasized  the 
importance  of  building  the  skill  levels  of  the  Iraqi  forces.  The  hope  is  that  when  U.S.  troops 
withdraw,  they  will  leave  behind  an  effective  local  workforce  and  a  secure  region.  He  also 
believes  that  creating  good  working  relationships  with  local  people  is  critical  for  the  immediate 
security  and  safety  of  the  U.S.  troops. 

You  are  called  in  when  conflicts  arise  between  the  U.S.  and  Iraqi  soldiers.  Lately  you’ve 
received  several  complaints  about  an  Iraqi  worker  who  has  been  harassing  one  of  the  U.S. 
soldiers. 


If  you  were  in  this  situation,  circle  (1-5)  how  important  it 

Very  Important  Not  Important 

would  be  to  know  what  is  bothering  the  Iraqi  worker? 

1  2  3  4  5 

While  the  Iraqi  knows  little  English,  he  has  been  using  deeply  offensive  language  to  describe  the 
mother  of  one  of  the  U.S.  soldiers.  For  the  U.S.  soldier,  this  insult  couldn’t  be  ignored.  What 
began  as  a  shouting  match  between  the  two  men  became  physical  with  both  men  throwing 
punches.  The  two  men  were  separated  and  ordered  to  report  to  your  office  where  an  interpreter 
will  be  available.  They  were  also  temporarily  removed  from  their  shift.  What  are  your  priorities 
in  handling  this  situation? 

Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  With  both  men  removed  from  their  shifts,  you  need  to  make  sure  that  there  are 
replacements  available  at  their  posts. 

b.  Take  a  position  between  the  two  men  in  order  to  lower  tensions  and  the  possibility  of 
additional  fighting. 

c.  Tell  the  men  that  you  know  that  it  is  difficult  to  work  together  with  no  shared 
language  and  you  believe  your  translator  may  help  resolve  the  problem. 

d.  Explain  that  their  conflict  cannot  continue.  Tell  them  you  need  their  help  to  settle  the 
conflict  so  that  you  don’t  have  to  make  an  official  report. 
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II.  Complex  Scenarios:  #3  Guard  Tower 


Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Maintain  security  during  this  disturbance. 

1— 

2 

-3— 

4 

—5 

Lower  tensions  and  additional  aggression. 

1— 

2 

-3— 

4 

—5 

Build  a  positive  relationship  between  the  two  soldiers. 

1— 

2 

-3— 

4 

—5 

Get  the  combatants  to  cooperate  with  you. 

1— 

2 

-3— 

4 

—5 

Both  the  American  and  Iraqi  are  hesitant  to  talk,  but  eventually  you  find  out  the  whole  story.  It 
turns  out  that  the  Iraqi  soldier  speaks  little  English.  In  his  effort  to  learn  English,  he  had  picked 
up  an  English  phrase  that  U.S.  soldiers  sometimes  use  jokingly  with  each  other.  He  did  not  know 
that  it  was  an  insult  that  could  not  be  tolerated  coming  from  an  outsider.  It’s  good  that  you 
solved  this  problem  because  it  could  have  interfered  with  your  base’s  security  or  developed  into 
something  bigger. 

Scenario  Continues 

The  next  evening,  you  are  faced  with  another  very  different  problem.  An  Iraqi  in  one  of  the 
guard  towers  sounded  the  alarm  to  report  movement  near  the  base  perimeter.  The  Iraqis  are  often 
“spooked”,  and  sound  the  alarm  even  when  there  isn’t  a  real  threat.  Not  only  is  this  stressful,  it 
also  reduces  the  urgency  of  legitimate  alarms.  This  has  become  a  recurring  and  potentially 
dangerous  situation. 

You’ve  tried  to  explain  that  alarms  should  be  sounded  only  in  case  of  an  enemy  sighting. 
However,  this  afternoon  an  Iraqi  signaled  an  alarm  when  he  saw  a  shepherd  herding  goats  too 
close  to  the  perimeter.  You  had  hoped  the  guards  could  distinguish  between  an  innocent  villager 
searching  for  a  stray  goat  and  an  insurgent  planting  an  IED.  Maybe  he  only  saw  movement  and 
felt  it  was  important  to  respond  as  quickly  as  possible.  Maybe  he  wanted  his  superiors  to  know 
that  he  was  able  to  spot  every  sign  of  trouble.  This  problem  is  very  frustrating,  often  a  waste  of 
time,  and  potentially  dangerous.  Given  this  background,  how  should  you  handle  this  particular 
false  alarm  and  the  problem  of  false  alarms  overall? 
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II.  Complex  Scenarios:  #3  Guard  Tower 


Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Continue  the  standard  protocol  and  treat  every  alarm  as  a  real  threat.  Request  that 
this  soldier  be  replaced  with  someone  capable  of  judging  danger. 

b.  Tell  the  Iraqi  soldier  you  value  that  he  is  working  hard  to  keep  the  base  safe. 

Require  that  he  re-take  his  training  so  that  he  can  keep  his  job  and  make  better  risk 
assessments  in  the  future. 

c.  Tell  the  Iraqi  soldier  that  you  know  their  task  can  be  difficult.  Express  your 
appreciation  for  his  work  and  explain  that  you  have  hopes  for  improvements. 

cl.  Have  his  Iraqi  supervisor  work  with  him  to  moderate  his  tendency  to  overreact  to 
non-critical  events. 

Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Maintain  effective  security  while  allowing  soldiers  to  treat 
every  alarm  as  a  real  threat. 

1  2  3  4  5 

Have  the  soldier  reach  acceptable  reporting  standards  by 
telling  him  you  respect  his  effort  but  that  he  must  repeat  the 
training. 

1  2  3  4  5 

Have  the  soldier  reach  acceptable  reporting  standards  by 
expressing  your  appreciation  for  the  soldier’s  work  and 
hoping  his  work  improves. 

1  2  3  4  5 

Reduce  the  Iraqi  soldiers  tendency  to  overact  using  the  Iraqi 
supervisors  help. 

1  2  3  4  5 

End  Scenario  #3:  Guard  Tower 


II.  Complex  Scenarios:  #4  Disaster  Relief 


Scenario  #4:  Disaster  Relief  In  The  Philippines 


Background 

The  U.S.  Pacific  Command  sent  a  task  force  to  assist  in  relief  efforts  following  a  flood  in  the 
Philippines.  The  storm  killed  hundreds  of  people  and  also  destroyed  roads,  schools,  and  other 
infrastructure.  The  immediate  impact  created  a  serious  food  and  water  shortage  causing 
dangerous  conditions  for  epidemics.  U.S.  fixed-wing  and  tilt-rotor  aircrafts  were  sent  in  to 
provide  crucial  access  to  the  remote  and  inaccessible  area. 

As  supplies  arrive  by  air,  people  stream  towards  hastily  set  up  distribution  centers.  You  are  the 
Staff  Sergeant  in  charge  of  the  food  distribution  for  one  village.  People  are  waiting  their  turn 
even  though  it  is  taking  several  hours  to  get  to  the  front  of  the  line. 

As  the  day  progresses,  the  lines  grow  longer.  You  notice  agitation  at  the  back  of  the  line.  You 
take  three  soldiers  and  a  translator  and  go  to  the  disturbance.  You  see  that  men  from  a 
neighboring  community,  armed  with  short  swords,  are  threatening  the  local  villagers.  The  local 
villagers,  understandably  terrified,  are  dropping  their  supplies  and  running. 


If  you  were  in  this  situation,  circle  (1-5)  how  important  it 

Very  Important  Not  Important 

would  be  to  know  what  the  looters  are  really  thinking? 

1  2  3  4  5 

The  looters  may  be  taking  supplies  because  their  own  village  has  not  yet  received  help.  People 
will  often  go  to  great  effort  to  care  for  their  families  and  their  friends.  On  the  other  hand,  they 
may  be  using  this  crisis  to  settle  long-standing  disagreements  or  to  acquire  new  territory  and 
objects  of  value. 

Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Attempt  to  identify  the  reason(s)  for  their  behavior  so  you  can  find  a  solution. 

b.  Give  the  looters  a  way  out  with  the  help  of  your  translator:  “this  situation  hasn’t 
gotten  out  of  control,  yet.  Leave  now  and  you  will  not  be  harmed.” 

c.  Explain,  with  the  help  of  your  translator,  that  you  would  like  to  find  a  solution  but 
that  first  they  must  return  all  of  their  weapons  to  their  village  or  leave  them  with  you. 

d.  Prepare  to  fire  non-lethal  shots  at  the  men  wielding  the  swords,  as  they  have 
potentially  deadly  weapons  and  you  cannot  risk  innocent  lives. 
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II.  Complex  Scenarios:  #4  Disaster  Relief 


Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Identify  the  looter’s  motivations  to  create  a  solution. 

1  2  3  4  5 

Get  the  looters  to  leave  immediately  by  letting  them  know  that 
if  they  do,  they  will  not  be  harmed. 

1  2  3  4  5 

Have  the  looters  return  their  weapons  or  leave  them  with  you 
in  exchange  for  the  chance  of  negotiating  a  compromise. 

1  2  3  4  5 

Secure  the  area  using  force. 

1  2  3  4  5 

Scenario  Continues: 

You  were  able  to  solve  the  looter  problem  and  the  relief  effort  is  now  going  smoothly.  The 
following  week,  however,  you  receive  a  report  from  mission  headquarters  that  samples  of  water 
from  the  local  wells  showed  signs  of  contamination.  You  are  ordered  to  seal  the  wells  and  carry 
out  a  NEO  (non-combatant  evacuation  operation)  moving  the  villagers  to  a  relocation  center  set 
up  about  12  miles  away. 

The  trucks  are  to  arrive  in  the  morning  and  you  issue  emergency  water  to  all  villagers  for  use 
during  the  relocation.  Some  residents  become  upset  and  protest.  They  are  afraid  that  the  looters 
will  return  to  steal  and  to  destroy  their  village.  They  love  their  homes  and  don’t  want  to  leave 
them  even  briefly.  How  might  you  best  accomplish  the  evacuation  with  the  help  of  your 
translator? 

Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Talk  with  the  villagers  to  assure  them  you  are  taking  steps  to  keep  their  village  safe. 

b.  Tell  the  villagers  that  the  bad  water  can  sicken  children.  Tell  them  that  you  have  a 
young  son  and  know  how  much  they  must  want  to  keep  their  children  safe. 

c.  When  the  trucks  arrive,  assign  families  to  the  trucks.  If  people  resist,  use  the  local 
leaders  to  insure  every  person  is  on  a  truck.  Use  force  if  necessary. 

cl.  Help  them  understand  the  danger  by  comparing  the  color  of  clean  water  with  water 
from  their  well.  Show  pictures  of  the  harmful  effects  of  contaminated  or  ‘dirty’  water. 
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II.  Complex  Scenarios:  #4  Disaster  Relief 


Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Calm  the  villagers  by  explaining  the  safeguards  you  have  in 
place. 

1  2  3 - 4  5 

Gain  acceptance  of  the  evacuation  by  explaining  that  you,  like 
them,  are  concerned  about  the  well  being  of  their  children. 

1  2  3  4  5 

Use  local  enforcement  leaders  to  move  people  into  trucks  to 
complete  your  assignment. 

1  2  3  4  5 

Convince  the  residents  that  the  water  is  unclean. 

1  2  3  4  5 

End  Scenario  #4:  Disaster  Relief 


Scenario  #5:  Bosnian  Weapons  Harvest  (Peacekeeping) 


Background 

Bosnia  had  experienced  heavy  damage  during  earlier  and  long-standing  conflicts.  Many  people 
still  have  firearms  in  their  homes.  To  create  a  safer  community,  the  U.S.  military  designed  a  two- 
phase  weapons  collection  program.  During  Phase  1,  the  U.S.  team  opened  a  local  drop-off  site 
where  people  could  dispose  of  weapons  and  meet  the  U.S.  peacekeepers.  For  Phase  2,  every 
household  received  a  pamphlet  in  the  mail  announcing  door-to-door  visits  to  make  it  easier  for 
people  to  dispose  of  weapons.  The  pamphlets  stressed  that  the  safety  was  the  main  concern  and  it 
encouraged  people  to  give  up  their  weapons. 

Y ou  are  leading  one  of  the  weapons  collection  teams  visiting  homes  and  encouraging  people  to 
dispose  of  weapons.  You  approach  each  home  with  two  soldiers  and  a  local  translator.  Many 
people  say  they  do  not  have  any  firearms  but  most  visits  are  friendly.  At  one  home,  however,  an 
agitated  man  opens  the  door.  Your  translator  asks  him  if  he  has  any  weapons  he’d  like  to  give  to 
your  team.  The  man  becomes  very  hostile  and  says  that  he  has  to  leave  for  work  immediately. 
You  ask  if  he  has  weapons  where  he  works  and  suggest  that  he  might  also  dispose  of  these.  The 
man  ignores  the  question,  walks  into  his  house  and  emerges  carrying  work  tools. 


If  you  were  in  this  situation,  circle  (1-5)  how  important  it 

Very  Important  Not  Important 

would  be  to  understand  what  is  troubling  the  man? 

1  2  3  4  5 

Your  priorities:  How  important  is  each  of  these  actions?  Review  all  of  the  actions  below  ancl 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once. 


Rank 

Actions 

a.  Make  conciliatory  gestures  to  put  the  man  at  ease.  Tell  him  that  he  is  not  required  to 
give  up  his  weapons  but  that  your  job  is  to  help  him  do  so  if  he  is  interested. 

b.  Record  the  location  of  the  man’s  home  to  keep  an  eye  out  for  potential  threats.  After 
this,  move  on  to  the  next  house. 

c.  Point  out  a  child’s  bike  in  the  yard  and  have  your  translator  explain  the  potential 
danger  of  a  weapon  to  neighborhood  children. 

d.  Have  your  team  to  return  to  your  vehicle.  Have  your  translator  explain  to  the  man 
that  you  are  not  there  to  cause  trouble  but  to  help  make  his  community  safer. 
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Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Make  a  conciliatory  gesture  that  would  put  the  man  at  ease. 

1  2  3  4  5 

Use  the  situation  to  increase  your  attention  to  potential 
neighborhood  problems. 

1  2  3  4  5 

Persuade  the  man  to  surrender  his  weapons  voluntarily  for 
the  sake  of  neighborhood  children. 

1  2  3  4  5 

Have  your  translator,  a  local  person,  speak  with  the  man  to 
help  him  see  the  danger. 

1  2  3  4  5 

Scenario  Continues: 

As  you  continue  around  the  village,  a  boy  comes  out  of  his  home  and  unexpectedly  approaches 
you  with  a  grenade  in  his  hand.  You  look  at  it  and  use  gestures  to  explain  to  the  boy  exactly  how 
he  should  hand  the  object  to  one  of  your  soldiers.  The  boy  follows  this  request  and  places  the 
grenade  as  directed.  The  soldier  locates  a  place  behind  a  brick  wall  that  is  a  safe  distance  away. 
You  then  get  on  your  radio  and  report  your  position  and  the  condition  of  the  grenade.  You  are 
told  that  the  EOD  (Explosive  Ordnance  Disposal)  team  will  arrive  on  scene  to  help  ASAP.  You 
must  wait  for  EOD  to  arrive.  You  are  uncertain  when  they  will  come  so  you  move  to  a  safe 
location.  Many  children  and  adults  notice  your  presence  in  the  village  and  begin  to  approach 
you.  They  are  curious  why  you  stopped.  What  do  you  do? 

Your  priorities:  How  important  is  each  of  these  actions?  Review  cdl  of  the  actions  below  and 
rank  them  from  ‘1  ’for  most  important  to  ‘4’  for  least  important.  Use  each  number  only  once 


Rank 

Actions 

a.  Respond  calmly  to  the  gathered  people  to  gain  co-operation.  Assure  those  in  the  area 
that  steps  for  safe  removal  of  the  grenade  have  been  initiated. 

b.  The  grenade  is  a  high  threat  to  your  team  and  the  villagers.  Have  your  team  cordon 
off  the  area  around  the  grenade  in  order  to  ensure  safety  for  your  team  and  villagers. 

c.  As  long  as  you  must  wait,  chat  with  the  adults  who  have  gathered.  If  kids  in  the 
village  play  soccer,  challenge  them  to  a  friendly  game  at  a  safe  distance  from  the 
grenade. 

cl.  Use  this  grenade  incident  to  express  the  dangers  of  having  weapons  in  homes. 

Explain  to  the  villagers  that  you  can  dispose  of  weapons  safely  if  they  bring  them  to 
you. 
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Your  confidence:  If  you  were  in  this  situation,  circle  how  confident  you  would  be  that  you  could 
accomplish  each  action  (1  =  Very  Confident,  5  =  Not  Confident). 


How  confident  would  you  be  that  you  could: 

Very  Confident  Not  Confident 

Provide  reassurance  to  the  citizens  and  maintain  calm 
behavior  in  the  area. 

1  2  3 - 4  5 

Cordon  off  the  area  to  keep  your  team  and  the  villagers  safe. 

1  2  3  4  5 

Use  the  opportunity  to  build  good  relationships  by  initiating 
recreational  activity  with  the  children  and/or  chatting  with 
adults. 

1  2  3  4  5 

Use  this  dangerous  situation  to  convince  the  people  to  hand 
over  their  weapons. 

1  2  3  4  5 

End  Scenario  #5:  Bosnian  Weapons  Harvest 


Please  wait  for  directions  to  begin  the  next  section. 


III.  Accomplishing  Goals 


III.  Accomplishing  Goals 


During  overseas  deployments,  soldiers  may  interact  with  people  from  different  cultures.  In 
this  final  section  you  will  see  situations  from  the  earlier  scenarios.  For  each  situation,  mark 
the  good  response(s)  with  a  checkmark  (V)  and  mark  the  bad  response(s)  with  an  (X).  For 
each  question,  use  V'  or  X  as  many  times  as  needed. 

I.  In  Broadsided ,  an  Iraqi  taxi  was  in  an  accident  with  an  Army  truck.  Iraqis  began  gathering 
and  shouting  at  the  U.S.  soldiers  in  a  busy  traffic  circle. 

1)  How  could  you  learn  the  cause  of  the  Iraqi  crowd’s  reaction? 

_ Ask  people  in  the  crowd  about  the  accident. 

_ Think  about  how  you  would  react  if  your  car  had  been  damaged. 

_ Review  your  past  experiences  with  Iraqis  and  look  for  similar  examples. 

_ Ask  the  U.S.  soldiers  on  site  why  the  crowd  reacted  as  it  did. 

2)  What  are  effective  ways  to  calm  down  the  crowd? 

_ Address  the  crowd  quickly  before  tensions  rise. 

_ Raise  your  voice  and  order  the  crowd  to  calm  down. 

_ .  Isolate  SGT  Lawrence  so  he  can’t  increase  tension. 

_ Raise  your  weapons  so  they  are  visible  to  the  crowd. 

3)  How  might  you  build  a  good  relationship  with  Walid? 

_ Promise  him  that  he  will  receive  money,  even  though  he  might  not. 

_ Tell  Walid  that  you  understand  his  frustrations  about  his  taxi. 

_ Tell  him  that  intentional  crashes  are  common  so  he  may  be  in  trouble. 

_ Tell  Walid  “we’re  both  better  off  if  this  is  solved  quickly.” 

4)  How  might  you  enlist  Walid  to  help  reduce  the  tension? 

_ Tell  Walid  that  you  will  help  him  but  he  needs  to  help  calm  the  crowd. 

_ Explain  that  when  things  settle  down,  you’ll  help  him  file  for  damages. 

_ Tell  Walid  that  if  he  doesn’t  cooperate,  his  taxi  services  will  no  longer  be  welcomed 

on  the  base. 

_ Control  the  situation  by  repeat  your  position  until  he  understands  and  yields. 
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III.  Accomplishing  Goals 


II.  In  the  Shots  into  the  Crowd  incident,  after  an  initial  meeting  with  the  town  council  you 
experienced  major  problems. 

1)  How  could  you  build  a  good  relationship  with  the  council? 

_ Ask  the  council  leaders  about  their  city’s  needs  and  refer  to  these  during  the  meeting. 

_ Explain  how  much  they  could  learn  from  the  U.S.’s  success. 

_ Tell  the  council  about  your  priority  for  the  safety  of  your  men. 

_ Explain  why  you  and  your  team  are  there  and  how  you  plan  to  help  them. 

2)  How  would  you  figure  out  what  the  council  members  are  most  concerned  about? 

_ Compare  this  situation  to  similar  situations  in  your  past. 

_ Ask  the  Iraqi  council  members  for  their  thoughts  and  opinions  directly. 

_ Think  about  the  concerns  you  would  have  living  in  this  town. 

_ Talk  with  soldiers  who  have  worked  successfully  with  Iraqis  in  the  past. 

3)  After  the  council  meeting,  three  children  were  shot  by  your  team.  What  would  be 
appropriate  ways  to  calm  this  situation? 

_ Leave  the  situation  immediately  as  your  presence  may  add  to  the  tension. 

_ Arrange  for  the  injured  child  to  be  medevac’d  to  a  hospital  quickly. 

_ Have  your  troops  keep  a  security  posture.  If  approached  tell  people  to  back  off  -  no 

exceptions. 

_ Lower  your  weapons  to  reduce  hostilities. 

4)  After  the  shootings,  how  would  you  convince  people  to  help  manage  the  situation? 

_ Ask  that  they  give  your  team  space  to  provide  immediate  medical  care. 

_ Strengthen  your  security  posture  and  order  the  town  people  to  leave  the  area. 

_ Describe  how  the  people  shared  responsibility  by  making  your  soldiers  nervous. 

_ Ask  people  living  near  to  bring  water  and  supplies  to  help  the  children. 
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III.  Accomplishing  Goals 


III.  Think  back  to  the  Disaster  Relie  f  scenario  when  contaminated  water  meant  you  had  to  move 
the  people  to  temporary  shelters.  Some  villagers  were  afraid  and  refused  to  go.  They  feared  that 
their  aggressive  neighbors  would  return  and  destroy  their  village. 

1)  How  would  you  convince  the  villagers  to  move? 

_ Remind  them  of  the  good  treatment  they  have  received. 

_ Order  them  onto  the  trucks  using  force  if  necessary. 

_ Tell  them  you  won’t  defend  the  village  unless  they  leave. 

_ Explain  the  danger  posed  by  the  water  and  the  protection  provided  to  their  homes. 

2)  During  the  relocation  one  villager  demands  to  be  returned  home.  What’s  the  best  way  to 
calm  the  situation? 

_ Speak  with  the  villager  and  subtly  suggest  that  the  delay  he  is  causing  may  expose 

the  rest  of  the  villagers  to  danger. 

_ Subdue  the  villagers  demands  by  threating  him. 

_ Announce  to  the  rest  of  villagers  that  he  is  endangering  everyone’s  safety. 

_ Move  him  out  of  view  of  the  others  and  talk  quietly  with  him. 


IV.  Think  back  to  the  Bosnian  Weapons  Harvest  scenario. 

2)  How  would  you  figure  why  the  man  was  hostile  when  you  knocked  at  his  door? 

_ Talk  with  the  man  at  a  more  relaxed  time. 

_ People  are  often  agitated  when  they’re  frightened  so  this  is  the  likely  cause. 

_ Ask  your  translator,  who  is  native  to  the  area,  why  he  thinks  the  man  is  agitated. 

_ Think  about  how  you  would  react  in  this  situation. 


3)  How  would  you  build  relationships  with  the  Bosnians  while  you  wait  for  EOD  to  take 
care  of  the  grenade? 

_ Suggest  a  soccer  game  with  the  young  people  to  build  trust. 

_ Tell  the  crowd  that  the  problem  stems  from  the  presence  of  weapons  in  homes 

_ Ask  the  adults  about  their  community  and  accept  offers  for  tea  while  you  wait. 

_ Stay  apart  from  the  villagers  to  respect  their  privacy. 


Please  continue  to  the  Demographic  Information  section  on  the  next 

page. 
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Demographic  Information 


Demographic  Information 


1.  Your  current  rank? 

Number  of  Years: 

0-2 

3-5 

6-10 

10-15 

16+ 

2.  Years  of  service  at  current  rank 

□ 

□ 

□ 

□ 

□ 

3.  Years  of  total  active  duty  service 

□ 

□ 

□ 

□ 

□ 

4.  Have  you  had  intercultural  training? 

If  so,  please  describe: 

□  Yes  □  No 

5.  Sex: 

□  Male  □Female 

6.  Age: 

17-20 

□ 

21-24 

□ 

25-28 

□ 

29-34 

□ 

35+ 

□ 

7.  Years  of  post  high  school  education: 

8.  Longest  Held  Civilian  Job: 

Position  Title  &  Duration  (months): 

10.  Languages 

First : 

Second: 

Please  list  below  the  places  you  have  been  deployed  and  the  total  number  of  months  you  were  deployed  there. 


Number  of  Months: 

0-6 

7-12 

13-18 

19-24 

25+ 

Iraq 

□ 

□ 

□ 

□ 

□ 

Afghanistan 

□ 

□ 

□ 

□ 

□ 

East/Southeast  Asia 

□ 

□ 

□ 

□ 

□ 

Central/South  America 

□ 

□ 

□ 

□ 

□ 

Other: 

□ 

□ 

□ 

□ 

□ 

Other: 

□ 

□ 

□ 

□ 

□ 

Other: 

□ 

□ 

□ 

□ 

□ 

Please  continue  to  the  Final  Consent  &  Authorization  Section  on  the  next  page. 
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Final  Consent  &  Authorization 


Final  Consent  &  Authorization 


Do  you  authorize  MacroCognition  LLC  to  compile  and  analyze  the  responses 
you’ve  provided  in  this  Gap  Study  packet? 

□  YES 

□  NO 

(Please  check  one  box) 


We  thank  you  for  your  participation.  We  hope  that  the  information  you  have 
provided  will  help  future  members  of  the  U.S.  military  work  more  effectively  and 
safely.  If  you  have  any  questions  or  concerns,  please  let  us  know.  If  you  can’t 
contact  us  before  we  leave,  our  contact  information  is  on  the  debriefing  form. 

-  The  MacroCognition  Research  Team 
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Summary  of  Gap  Study  data  collection  activity  at  Fort  Benning 
Date:  3  July  2014 

Researchers:  Helen  Altman  Klein  (MacroCognition  LLC),  Joseph  Borders  (MacroCognition 
LLC)  &  Scott  Flanagan  (Sophia  Speira) 

Overview:  MacroCognition  LLC  conducted  data  collection  for  the  Gap  Study  on  3  July  2014  at 
Fort  Benning,  Georgia.  Over  160  Non  Commissioned  Officers  enrolled  in  the  Henry  Caro  NCO 
Academy  (Leader  Development)  completed  this  summative,  three  part  study. 

The  Gap  Study  contains  scenario  driven  data  collection  methods  developed  in  TA IB,  as  well  as 
more  standard  survey  and  interview-based  methodologies.  This  study  is  designed  to  identify  the 
cognitive,  behavioral,  and  perceptual  gaps  between  the  Good  Stranger  (GS)  skills  needed  and 
those  possessed  by  participants.  The  present  work  supports  Raytheon  BBN  Technologies  current 
effort  in  developing  a  social  training  simulator.  First,  it  will  provide  a  Frame  Assessment  ( Brief 
Scenarios ):  to  diagnose  the  prominence  of  four  cognitive  frames  or  mental  models  -  Good 
Stranger,  Mission,  Rules  &  Procedures,  and  Authority /Domineering.  We  hope  to  validate  this 
cost  effective  tool  as  an  effective  option  for  identifying  the  personnel  who  would  benefit  most 
from  the  BBN  simulator.  Second,  this  research  will  provide  a  Learning  Objectives  Assessment 
( Complex  Scenarios ):  to  diagnose  decision  priorities  in  complex  military  situations.  This  section 
provides  participants’  preference  and  confidence  in  performing  actions  related  to  Learning 
Objectives  we’ve  identified  as  most  conducive  to  the  GS  mindset  -  de-escalation,  voluntary 
compliance,  building  rapport,  perspective-taking,  and  security.  Lastly,  we  will  employ  a  final 
Ability  Assessment  ( Accomplishing  Goals):  examines  how  accurately  participants  understand 
the  behaviors  that  support  the  learning  strategies  they  encountered  in  the  previous  section.  These 
three  profiles  can  guide  trainee  selection  and  individualize  training  to  best  support  the 
development  of  social  interaction  skills  and  human  dynamics  proficiencies  for  future  trainees. 

Schedule:  We  collected  data  with  161  Non-Commissioned  soldiers  enrolled  in  the  Henry  Caro 
Academy  (Infantry  Senior  Leader  Course)  on  3  July  2014  from  0700  -  0840. 

0700  -  0715  Greetings,  Introduction,  Consent  From  (Helen  Altman  Klein) 

0715  -  0730  Frames  (Brief  Scenarios) 

0730  -  0810  Learning  Objectives  (Complex  Scenarios) 

0815  -  0825  Ability  (Accomplishing  Goals) 

0825  -  0840  SSIM  Briefing,  Thank  You  (Scott  Flanagan) 

Location:  USARMY,  Fort  Benning,  GA  -  NCOA  Infantry  Senior  Leader  Course 
Building  #  224,  Room  #132 

Sponsor:  Funding  for  this  project  was  provided  by  the  SSIM  (Strategic  Social  Interaction 
Modules)  program  within  DARPA  (Defense  Advanced  Research  Projects  Agency)  BAA- 11-32 
Technical  Area  2. 

SSIM  goal:  In  addition  to  collecting  data  for  the  Gap  Study,  the  topic  of  the  scenarios  involved 
approaching  civilians  with  respect  and  attempting  to  build  trust,  as  opposed  to  trying  to  gain 


compliance  through  coercion  and  intimidation  which  might  work  in  the  short-term  but  appears  to 
generate  anger  and  a  desire  for  retaliation  in  the  long-term. 

Data  collection  plan:  Our  plan  was  to  run  120  enlisted  Army  soldiers  (a  mixture  of  E-5  -  E7). 
MacroCognition  LLC  provided  soldiers  with  a  consent  form  upon  their  arrival.  At  this  time,  the 
soldiers  were  made  aware  of  their  rights  as  research  participants,  and  they  could  opt  to  not  sign 
the  consent  form  without  any  consequence.  Next,  consenting  participants  completed  the  Gap 
Study  as  MacroCognition  LLC  personnel  facilitated  instructions.  After  the  completion  of  the 
study,  Scott  Flanagan  (Sophia  Speira)  briefed  the  participants  on  the  SSIM  initiative  and  the  Gap 
Study.  Participants  were  thanked  and  given  a  debriefing  form  with  similar  information.  The 
study  was  completed  by  0900. 

Process  outcome:  We  were  successful  in  collecting  all  the  data  we  needed.  In  fact,  we 
surpassed  our  expected  goal  of  120  participants.  We  appreciate  all  of  the  support  we  received  at 
Fort  Benning.  We  now  are  beginning  the  analysis  of  the  data,  which  will  take  many  weeks,  but 
we  wanted  to  use  this  memorandum  to  describe  our  project  and  to  express  our  gratitude. 

Informal  Evaluation:  Although  we  are  just  starting  to  sort  through  the  data,  we  are  beginning 
to  see  trends  emerge,  particularly  in  the  Frames  Assessment  (10  brief,  scenario-based,  ranked 
choice  questions).  We  are  starting  to  see  that  across  the  (10)  scenarios,  individuals  prefer 
different  frames  (e.g.  GS  orientation,  Mission  orientation).  Also,  our  preliminary  data  shows  that 
scenarios  may  impact  participants’  preferred  frame  as  well.  For  instance,  riskier  scenarios  may 
cause  participant’s  to  take  on  a  Rules  &  Procedure  or  Authoritarian  orientation,  rather  than  a  GS 
frame.  We  plan  to  continue  these  analyses  and  further  explore  how  the  population  varies  in 
prioritizing  our  Learning  Objectives  and  how  they  vary  in  self-efficacy  (confidence). 
Furthermore,  we  can  analyze  and  compare  prioritizing,  perceived  capability  and  actual 
ability /understanding  of  the  Learning  Objectives.  As  we  continue  to  analyze  our  data,  we  will 
distribute  our  findings  to  Fort  Benning,  BBN  and  the  SSIM  team. 

Prepared  by: 

Helen  Altman  Klein,  Ph.D.,  Senior  Scientist  at  MacroCognition  LLC 
helen @ macrocognition.com.  937/238-8281 

Joseph  Borders,  Research  Associate  at  MacroCognition  LLC 
ioseph@macrocognition.com.  937/631-6774 
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ABSTRACT 

US  military  personnel  encountering  civilians  can  function  as  “Good  Strangers”  (GS),  cultivating 
co-operation  and  safety,  or  they  can  act  in  ways  that  increase  hostility.  Current  work  investigates 
the  dynamics  of  social  interactions  and  training  technologies  attempting  to  increase  positive 
outcomes  of  military  social  encounters.  Soldiers  at  Ft.  Benning  completed  the  GS  Diagnostic  Tool 
measuring  the  GS  cognitive  frame  and  its  strategic,  behavioral  and  affect  components.  First,  we 
examined  differences  between  a  GS  frame  and  other  cognitive  frames:  Mission,  Rules/Procedures, 
and  Authority.  We  then  investigated  priorities  for  tactics  and  skills:  Perspective  Taking,  De- 
escalation,  Building  Rapport,  Voluntary  Compliance,  and  Security.  Finally,  we  assessed 
confidence  and  competence  levels  for  the  tactics  listed  above.  We  found  differences  among 
warfighters  as  well  as  significant  relationships  among  frame  preference,  response  tendencies,  and 
competencies.  The  research  suggests  how  social  interaction  training  might  be  most  effective  when 
addressing  gaps  between  initial  capacities  of  the  learner  and  GS  essentials. 

KEYWORDS 

Decision  Making,  Military,  Macrocognition,  Adaptive  Learning,  Counter  Insurgency 

INTRODUCTION 

Ongoing  advances  in  computer-based  instructional  technology  have  revolutionized  the  approaches  we  can  use  to 
deliver  training.  These  advances  have  increased  the  effectiveness  of  training  and  dramatically  reduced  the  time 
needed  to  achieve  positive  outcomes.  They  have  also  facilitated  training  for  more  complex  cognitive  capacities, 
such  as  cultural  awareness  and  social  competencies  (Raybourn,  2005).  One  reason  for  this  is  that  adaptive 
instruction  can  individualize  training  by  adjusting  to  the  learner’s  initial  state  and  by  continually  accommodating 
increments  in  their  knowledge/skill.  Learners  can  receive  a  stream  of  feedback  about  their  performance  to 
enhance  their  decision  making  capabilities  and  improve  their  understanding.  This  purpose  of  this  paper  is  to 
introduce  the  Good  Stranger  Diagnostic  Tool,  a  cognitive  and  social  competence  assessment  tool  as  a  prototype 
to  support  advances  in  adaptive  learning  and  training  in  the  military  sector. 

Good  Stranger  Project 

This  work  was  undertaken  as  part  of  MacroCognition  LLC's  involvement  in  the  Defense  Advanced  Research 
Projects  Agency  (DARPA)  Strategic  Social  Interactions  Module  (SS1M)  program.  This  program  was  designed 
to  investigate  the  dynamics  of  maintaining  a  Good  Stranger  (GS)  cognitive  frame  during  military  and  civilian 
encounters.  Because  of  the  unique  demands  facing  military  personnel  during  foreign  deployments,  we  needed  to 
go  beyond  the  expansive  literature  on  “soft”  methods  of  persuasion  and  influence  (e.g.,  Cialdini,  1993; 
Thompson  &  Jenkins,  1993/2004;  Glennon,  2010;  Wilson,  2011;  Thaler  &  Sunstein,  2008).  For  this  reason,  we 
conducted  Cognitive  Task  Analysis  (CTA)  interviews  with  over  forty  military  and  police  personnel  to  better 
understand  what  basic  skills  are  needed  to  foster  constructive  social  encounters,  regardless  of  language  and 
cultural  barriers,  in  high-risk  environments.  We  include  police  personnel  because  they  face  some  of  the  same 
challenges  as  warfighters.  In  the  process,  we  formulated  an  empirical  model  (see  Figure  1)  centered  on  the 
process  of  building  trust  (G.  Klein.  H.  A.  Klein.  Lande,  Borders  &  Whitacre,  2014b).  We  identified  behavioral 
and  cognitive  strategies  that  were  strong  predictors  of  building  trust  and  conducive  to  being  a  Good  Stranger, 
namely  building  rapport,  de-escalating,  perspective  taking,  and  preferring  voluntary  compliance.  These 
strategies  are  behavioral  extensions  of  the  Good  Stranger  Cognitive  Frame. 

Our  CTA  interviews  with  police  officers  and  military  personnel  also  identified  substantial  individual  differences 
in  preferences  for  taking  the  perspective  of  others,  building  rapport,  and  de-escalate  conflicts  (G.  Klein,  H.  A. 
Klein.  Lande,  Borders  &  Whitacre,  2014a).  The  GS  strategies  are  not,  of  course,  rule-based  or  procedural,  but 
depended  on  cognition  -  making  sense  of  complex  situation  and  creating  solutions  dependent  on  the  situation. 
The  interviewees  also  seemed  to  differ  in  their  skill  in  implementing  these  strategies.  Interviewees  who  showed 
greater  skill  in  implementing  these  strategies  were  more  likely  to  build  trust  with  civilians  and  community 
leaders.  These  cognitive  skills  and  the  receptivity  to  new  ideas  are  basic  to  the  Good  Stranger  cognitive  frame. 
Some  interviewees  appeared  more  equipped  to  build  trust  with  civilians  and  community  leaders  while  others  did 
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not  use  this  cognitive  frame  when  sizing  up  situations.  This  variability  makes  a  standardized  program  of 
instruction  difficult. 

Cognitive  Frames 

Instead  of  trust  building,  perceptions  and  behaviors  may  be  driven  by  other  underlying  goals.  Using  the 
data/frame  model  of  sensemaking  (G.  Klein,  Moon  &  Hoffman,  2006a,  2006b),  we  examined  how  military  and 
police  personnel  varied  in  the  way  they  made  sense  of  situations.  Based  on  our  CTA  interviews,  we  hypothesize 
that  respondents  will  vary  in  the  frames  they  use  to  make  sense  of  situations.  Depending  on  their  frame 
preference,  warfighters  will  notice  different  cues,  form  different  expectancies,  generate  different  goals  and 
consider  differing  courses  of  action  (see  Figure  1).  In  doing  this,  we  identified  three  prominent  cognitive  frames 
in  addition  to  the  GS  frame  that  military  personnel  use  to  make  sense  of  situations:  Rules  and  Procedures, 
Mission  and  Authority. 

Rules  and  Procedures  Frame 

The  military  as  an  organization  has  a  well-defined  hierarchical  leadership  and  centralized  decision  making  that 
promotes  structured  thinking.  Warfighters  are  trained  to  adhere  to  rules  and  procedures  and  rely  on  these 
guidelines  in  times  of  uncertainty.  Rules  and  Procedures  orient  warfighters  to  follow  protocol  and  avoid 
deviating  from  these  rules  as  their  highest  priority.  In  combat,  rules  and  procedures  are  critical,  as  they  maintain 
safety  and  security.  This  frame  can  be  used  in  co-ordination  with  other  frames,  like  the  Mission  frame. 
Warfighters  can  closely  proposed  guidelines  while  working  towards  mission  completion. 

Mission  Frame 

Some  warfighters  see  the  mission  as  the  driving  force  behind  decisions  and  actions.  They  do  not  rely  as  heavily 
on  a  Good  Stranger  or  Rules  and  Procedures  frame  to  gauge  situations.  Instead,  they  rely  on  creativity  and 
strategic  thinking  in  an  attempt  to  optimize  mission  completion.  The  Mission-oriented  approach  can  also  use 
other  frames  to  make  sense  of  the  situation  and  act  in  an  effective  manner  while  promoting  mission  completion. 

Authority  Frame 

Security  is  an  essential  component  of  the  warfighter’s  worldview.  However,  some  high  security-minded 
individuals  may  exaggerate  this  approach  and  become  domineering.  These  warfighters  use  their  authority  and 
power  to  gain  compliance  from  others  and  they  do  not  hesitate  to  use  coercion.  They  also  tend  to  escalate 
situations  because  they  take  provocations  personally.  The  Authority  frame  can  promote  resentment  and  distrust 
among  community  members,  and  sometimes  yields  unfavorable  long-term  consequences  for  military  and  police 
forces. 


Figure  1.  The  Good  Stranger  Frame:  To  Building  Trust 


These  frames  are  not  mutually  exclusive  and  situational  demands  influence  the  frame  being  used.  Kinetic 
encounters  involving  hostile  enemies  demand  an  Authority  frame  or  a  Rules  and  Procedure  frame.  For  example, 
the  cues,  goals,  expectancies  and  actions  differ  when  needing  to  protect  one’s  life  or  the  life  of  others  than  when 
wanting  to  gain  trust  with  community  leaders  and/or  civilians.  We  expect  that  all  participants  will  be  concerned 
with  self-protection  and  most  will  use  a  Mission  and/or  a  Rules  and  Procedure  frame.  Some  will  have  some  or 
most  elements  of  a  GS  frame.  Participants  with  GS  frames  will  differ  from  the  others  in  the  way  they  appreciate 
means  of  gaining  voluntary  compliance,  the  way  they  engage  in  perspective  taking,  and  the  way  they  react  to 
cultural  differences  -  being  more  sympathetic,  curious,  and  adaptable  as  opposed  to  being  judgmental  and 
ethnocentric. 
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The  Good  Stranger  Diagnostic  Tool 

Our  earlier  interviews  sugggested  that  a  scenario-based  diagnostic  assessment  will  be  able  to  support  training  in 
many  ways.  First,  it  will  identify  participants  with  the  potential  to  become  GSs.  This  tool  will  also  allow  us  to 
identify  the  cognitive  and  behavioral  “gaps”  between  the  participant  and  the  GS  frame.  These  gaps  will  then 
become  the  basis  for  individualizing  training.  The  tool  will  also  allow  the  assessment  of  the  knowledge  of 
tactics  and  skills  needed  for  effective  social  interaction  skills  -  perspective  taking,  rapport  building,  voluntary 
compliance  and  de-escalating  needed  to  increase  trust  during  social  interactions  (see  Table  2).  It  will  also  allow 
the  identification  of  critical  tactics  and  skills  that  are  needed  for  a  specific  soldier’s  training.  Finally,  the  tool 
will  allow  the  assessment  of  the  confidence  needed  to  impliment  these  tactics  and  skills.  All  of  this  information 
could  be  continually  updated  during  training.  From  the  learners  perspective,  they  will  always  be  moving 
forward  to  new  and  needed  content. 

On  the  other  end,  we  would  be  able  to  identify  those  people  whose  existing  cognitve  frame  and  behavioral 
patterns  are  simply  too  difficult,  costly,  or  even  impossible  to  alter  (e.g.  extremely  authoritarian).  The  Good 
Stranger  Diagnostic  Tool  was  designed  as  a  cost  effective  tool  for  identifying  the  personnel  who  would  benefit 
most  from  training,  particularly  training  that  is  resource  intensive.  These  profiles  can  guide  trainee  selection  to 
best  support  the  development  of  social  interaction  skills  and  human  dynamics  proficiencies  for  future  trainees. 
The  Good  Stranger  Diagnostic  Tool  depends  on  several  elements.  First,  we  must  be  able  to  assess  the  presence 
of  a  GS  Frame  and  the  understanding  of  the  GS  tactics  and  skills.  We  must  also  be  able  to  assess  if  the  learner 
knows  how  and  when  to  implement  these  skills,  rather  than  taking  a  security  approach.  Finally,  we  need  to 
know  if  the  learner  has  the  confideence  to  implement  the  skills  when  needed.  This  research  was  an  initial 
attempt  to  assess  the  likelihood  of  satisfying  these  requirements. 

METHOD 

Participants 

We  collected  data  from  non-commissioned  Army  soldiers  enrolled  in  the  Henry  Caro  Academy  (Infantry  Senior 
Leader  Course)  at  Ft.  Benning,  GA  ( N  =  78).  All  Soldiers  were  male,  ranked  E  6  -  Staff  Sergeant  (66%)  or  E  7 
-  Sergeant  First  Class  (34%),  and  had  at  least  one  overseas  deployment. 

Participants  reviewed  the  consent  form  and  research  personnel  explained  their  rights  as  research  participants  as 
defined  by  the  Institutional  Review  Board  (IRB).  Next,  participants  individually  completed  the  three-part  GS 
Diagnostic  tool  (described  in  full  below),  as  research  personnel  facilitated  each  section.  Upon  completion, 
participants  received  a  debriefing  form  and  short  briefing  explaining  the  SSIM  initiative  and  the  purpose  of  the 
GS  Diagnostic  Tool.  The  described  procedures  were  in  accordance  with  both  MacroCognition’s  IRB  procedures 
and  those  of  the  US  Army  Human  Research  Protection  Office  (HRPO). 

Research  Measures 

Cognitive  Frames  Assessment 

Participants  read  ten  brief  military-based  scenarios  and  responded  to  prompts  by  ranking  alternatives  (1  =  most 
important.  4  =  least  important).  For  each  question,  participants  could  only  indicate  one  option  as  their  (1)  most 
important,  and  so  on.  This  assessment  examined  the  prominence  of  four  cognitive  frames  (see  Table  1).  Each 
response  option  manifested  one  of  the  four  cognitive  frames. 

Table  1 

Cognitive  Frame  Types  and  Descriptions 

Good  Stranger  Tries  to  gain  trust;  understands  long  term  consequences  of  action 

Mission  Makes  decisions  that  optimize  mission  completion 

Rules  and  Procedure  Tries  to  follow  protocol  and  avoid  deviating  from  rules 

Authority  Uses  authority  to  gain  compliance;  does  not  hesitate  to  use  coercion 

Tactics  and  skills 

The  next  section  presented  five  complex  military-based  scenarios,  with  multiple  decision  points  interspersed 
througout  each  scenario.  This  section  required  participants  to  rank  response  priorities  and  confidence  levels  in 
performing  actions  related  to  tactics  and  skills  we’ve  identified  as  most  conducive  to  the  GS  mindset  - 
Perspective  Taking,  De-escalation,  Building  Rapport,  Voluntary  Compliance.  We  also  included  Security, 
because  this  approach  is  critical  for  warfighters,  and  cannot  be  ignored  (see  Table  2).  At  each  decision  point, 
participants  were  required  to  rank  order  (1  =  most  important,  4  =  least  important)  four  of  the  tactics  and  skills 
(De-escalation,  Building  Rapport.  Voluntary  Compliance  and  Security).  Participants  could  only  idicate  one  of 
the  options  as  their  (1)  most  important.  Perspective  Taking  was  assessed  at  each  decision  point  using  a  separate 
Likert  Scale  because  this  tactic  is  more  cognitive  based  and  was  not  easily  comparible  to  the  other  skills  listed. 
Next,  particpants  indicated  their  confidence  for  each  of  the  four  tactics  (excluding  Perspective  Taking)  and  skills 
using  a  Likert  Scale  (1  =  very  confident,  5  =  not  confident  at  all). 
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Table  2 


Learning  Objectives  and  Descriptions 


Perspective  Taking 

De-escalation 

Building  Rapport 
Voluntary  Compliance 

Security 


Sees  the  world  from  other’s  perspectives  and  uses  this  to  understand  the  actions 
of  others 

Reduces  the  intensity  of  situations  or  conflicts;  remains  calm  and  helps  others  to 
move  towards  calm 

Works  to  develop  a  positive  relationships  with  others  to  accomplish  tasks 

Seeks  voluntary  rather  than  coercive  compliance  by  understanding  others  and 
using  socials  skills 

Considers  security  as  most  importance  when  assessing  situational  demands 


Ability  Assessment 

The  last  section  examined  how  accurately  soldiers  understood  the  behaviors  that  supported  the  skills  and  tactics 
they  encountered  in  the  previous  section  (see  Table  2).  For  example,  to  assess  de-escalation,  participants  were 
asked  to  reflect  on  an  earlier  scenario  and  indicate  the  effective  and  ineffective  ways  to  accomplish  the  goals  (for 
example,  calm  down  a  crowd).  Soldiers  indicated  their  judgment  of  effective  and  ineffective  strategies  for 
achieving  goals  by  selecting  from  two  optimal  and  two  incorrect  choices.  Each  question  was  followed  by  a  list 
of  four  alternatives.  Based  on  a  participant’s  percent  correct,  they  were  assigned  an  overall  Ability  score  which 
indicates  their  competency  for  the  skills  and  tactics  listed  in  Table  2. 


RESULTS 

Baseline  Assessment  -  Cognitive  Frame  Preference 

Our  inital  objective  was  to  examine  this  sample’s  frame  preferences,  as  a  whole.  To  generate  a  basic  frame 
preference  outlook,  we  combined  individual  frame  rankings  across  each  of  the  ten  frame  questions.  Using  these 
overall  frame  preference  scores,  we  calculated  group  averages  for  each  cognitve  frame.  The  highest  possible 
score  was  40.  Good  Stranger  was  the  most  preferred  frame  type  (M  =  29.22,  SD  =  3.71)  across  all  ten  scenarios, 
followed  closely  by  Rules  &  Procedures  (M  =  28.58,  SD  =  3.71).  The  Mission  frame  was  ranked  lower  (M  = 
25.19,  SD  =  3.03)  and  Authoritarian  was  the  least  preferred  frame  type  overall  (M  =  17.03,  SD  =  3.91). 

Frame  Type  Predicts  Response  Tendencies 

Next,  we  looked  for  relationships  of  frame  preferences  and  prioritization  of  tactics  and  skills,  confidence  levels 
and  Ability  scores  for  the  tactics.  Are  response  tendencies  related  to  cognitive  frame  preference?  Because  this 
was  an  exploratory  study,  we  used  correlational  analyses  to  test  for  association  between  the  variables  listed  in 
Tables  1  and  2.  Later,  we  will  apply  these  associative  links  to  relevant  implications  for  training. 

Good  Stranger  Frame 

As  expected,  participants  preferring  GS  frame  were  more  likely  to  use  the  tactics  and  skills  related  to  building 
trust,  instead  of  security.  The  Good  Stranger  frame  was  associated  with  increased  priority  for  De-escalation,  r  = 
.25,  p  <  .05  and  Building  Rapport,  r  =  .32,  p  =  <  .01,  and  decreased  priority  for  Security,  r  =  -.38,  p  =  <  .01. 

Mission  Frame 

Similar  to  the  GS  Frame,  participants  that  prioritized  the  Mission  frame  higher  demonstrated  decreased  priority 
for  Security  responses,  r  =  -.28,  p  <  .05.  They  also  showed  increased  competence  for  the  GS  skills  and  tactics 
listed  in  Table  2.  r  =  .27,  p  <  .05. 

Authoritarian  Frame 

Participants  that  highly  prioritized  the  Authoritarian  frame  over-prioritized  security  responses,  rather  than  using 
GS  tactics  and  skills.  High  Authoritarian  frame  scores  were  associated  with  high  prioritzation  of  Security,  r  =  - 
.38,  p  <  .001.  The  Authoritarian  frame  was  significantly  correlated  with  decreased  priority  for  Perspective 
Taking,  r  =  -.23,  p  <  .05,  Building  Rapport,  and  lower  overall  Ability  Scores,  r  =  -.35.  p  <  .05. 

Rules  &  Procedures  Frame 

Interestingly,  participants  with  high  Rules  and  Procedures  frame  preference  reported  lower  confidence  ratings 
for  De-escalation,  r  =  -.28,  p  <  .05,  Voluntary  Compliance,  r  =  -.23,  p  <  .05,  and  Security,  r  =  -.27,  p  <  .05. 
These  individuals  were  not  confident  in  performing  any  skills  and  tactics. 


DISCUSSION 
Theoretical  Contributions 

The  results  of  this  study  further  confirm  the  link  between  the  GS  frame  and  certain  trust-building  tactics,  such  as 
building  rapport,  gaining  voluntary  compliance  and  de-escalation.  Using  short  scenario-based  assessments,  we 
can  identity  participants  that  prefer  to  use  this  frame  in  social  encounters.  GS  frame  preference  significantly 
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predicted  De-escalation  and  Building  Rapport,  and  negatively  predicted  excessive  Security.  Using  the  GS 
Diagnostic  Tool,  we  found  that  other  frame  types,  like  the  Authoritarian  frame,  significantly  predict  decreased 
prioritization  and  understanding  of  certain  trust-building  tactics,  such  as  De-escalation  and  Building  Rapport. 
These  relationships  are  consistent  with  previous  SS1M  research,  investigating  the  Good  Stranger  concept  and 
trust-building  tactics  (G.  Klein  et  al.,  2014a,  2014b). 

Training  Implications 

The  Good  Stranger  Diagnostic  Tool  can  provide  a  description  of  the  learner’s  initial  cognitive  frame,  response 
tendencies,  and  understanding  for  certain  behavioral  tactics.  Adaptive  learning  technologies,  such  as  serious 
game  simulators,  can  use  this  information  to  generate  individualized  learning  conditions  in  order  to  augment 
training  effectiveness.  More  specifically,  using  the  cognitive  frame  assessment,  we  believe  training  systems  can 
modify  initial  starting  conditions  and  how  training  content  can  be  presented.  This  can  be  accomplished  by 
presenting  ability-graded  examples  and  or  additional  practice  as  needed.  We  propose  the  following  training 
recommendations  based  on  our  empirical  findings. 

While  some  warfighters  can  switch  frames  depending  on  the  situation  they  encounter,  others  may  be  unable  to 
adopt  a  GS  frame.  Military  personnel  that  prefer  a  domineering  approach  may  need  explicit  behavioral  rule- 
based  approaches  to  reduce  provocations  when  they  are  in  contact  with  civilians.  Participants  that  preferred  the 
Authoritarian  frame  were  more  likely  to  prioritize  security  over  the  social  tactics  and  skills,  regardless  of  the 
situational  demands.  Specifically,  our  results  indicate  they  were  less  likely  to  prioritize  Perspective  Taking  and 
Building  Rapport.  Both  of  these  dimensions  are  strongly  linked  to  empathy,  the  ability  to  recognize  and  share 
other’s  feelings,  which  Field  Manual  6-22  Army  Leadership  indicates  as  a  critical  tool  in  winning  support  from 
populations  (U.S.  Army,  2006).  Those  who  prefer  the  Authoritarian  frame  might  best  be  directed  to  positions 
that  do  not  require  interactions  with  civilians.  Instead,  these  warfighters  would  be  better  suited  in  positions 
involving  clearly  defined  rules  such  as  defense  and  security. 

Participants  preferring  the  Rules  and  Procedure  frame  showed  lower  confidence  ratings  for  social  tactics  and 
skills  (e.g.  De-escalation  and  Voluntary  Compliance)  and  security-based  items.  Military  personnel  that  strongly 
prefer  a  Rules  and  Procedure  frame  may  benefit  from  feedback  supporting  confidence  development  when 
needed.  This  feedback  can  provide  warfighters  with  the  positive  assurance  needed  when  making  effective 
decisions  regarding  security  and  social  tactics  (e.g.  building  trust). 

Those  preferring  the  GS  and/or  Mission  frame  demonstrated  similar  response  patterns.  Both  highly  prioritized 
the  GS  social  tactics  and  skills,  and  these  participants  also  showed  increased  understanding  for  the  GS  social 
skills.  Because  of  this,  warfighters  preferring  these  frames  would  benefit  from  accelerated  training. 
Importantly,  this  training  would  also  require  trainees  to  make  decisions  about  appropriate  times  to  initiate  GS 
tactics  vs.  taking  a  security  oriented  approach  and  vice  versa. 

Limitations 

Further  iterations  are  needed  to  construct  effective  frame  scenarios  and  response  options.  To  our  surprise,  most 
participants  preferred  the  GS  frame  when  making  sense  of  situations.  While  this  may  be  reflective  of  our 
sample,  we  are  skeptical  that  this  represents  the  population  as  a  whole.  We  suspect  some  participants  were  able 
to  see  patterns  in  the  response  options  and  attempted  to  respond  how  they  should  rather  than  how  they  actually 
would.  These  demand  characteristics  can  be  avoided  by  creating  credible  options  that  are  not  as  obviously  right 
or  wrong. 

This  sample  included  non-commissioned  officers  ranking  E6-E7  in  the  US  Army.  The  outcomes  provide  a 
specific  frame  of  reference  (e.g.  diagnostic)  for  this  group.  Commissioned  officers  would  potentially  generate 
different  response  patterns,  frame  preferences,  priorities,  confidence  levels  and  abilities.  More  testing  would  be 
needed  to  describe  potential  group  differences. 

The  sample  included  students  enrolled  in  a  senior  leadership  course.  Data  collection  occurred  when  all 
participants  had  a  major  test  in  their  senior  leadership  course  scheduled  later  in  the  day.  Due  to  scheduling 
issues,  participants  had  less  time  than  we  anticipated  to  complete  the  study  (1.5  instead  of  2  hours). 

In  order  to  accurately  assess  outcomes,  we  included  only  participants  who  had  completed  every  question  for 
each  segment.  Although  over  161  participants  attended  the  study  session,  only  about  two-thirds  of  the  sample 
completed  all  three  sections  of  the  GS  Diagnostic  Tool.  Despite  these  problems,  we  identified  important  training 
implications. 

The  GS  Diagnostic  Tool  was  also  able  to  diagnose  individual  frame  preferences,  response  tendencies  and  levels 
of  confidence  and  ability  for  specific  tactics  and  skills.  This  information  can  assist  computer  based  instructional 
systems  to  deliver  effective  training,  accommodating  individual  ability  strengths  and  limitations.  While 
additional  research  is  needed,  particularly  exploring  the  validity  of  the  measures  used,  we  believe  some 
components  of  our  study  can  be  incorporated  into  ongoing  training  efforts  to  augment  effectiveness. 
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Good  Stranger  Diagnostic  Tool  -  Addendum  A: 

The  following  training  recommendations  are  based  on  a  sample  of  78  non-commissioned  officers 
(E6-7)  enrolled  in  a  NCO  Henry  Cairo  Leadership  Academy,  stationed  at  Ft.  Benning,  Georgia. 
This  data  is  meant  to  provide  military  leadership  and  simulation  teams  with  answers  about  how 
this  sample  responded  to  the  social  skills  outlined  by  the  SSIM  team.  We  have  aggregated  group 
response  averages  to  provide  the  reader  with  overarching  themes  and  trends  from  our  data.  We 
caution  generalizing  these  findings  to  all  military  personnel,  NCOs,  or  even  E6-E7s.  To  make 
these  assumptions,  we  would  need  to  develop  a  more  comprehensive  diagnostic  tool,  and 
construct  a  more  rigorous  study  design.  However,  we  do  believe  these  findings  can  provide 
direction  for  military  leaders  and  trainers  developing  customized  training  programs  for 
warfighters. 

Cognitive  Frames 

The  type  of  cognitive  frame  a  person  employs  significantly  influences  how  they  perceive  their 
environment.  A  frame  will  affect  the  type  of  cues  we  spot,  what  expectancies  we  generate  and 
which  goals  we  attempt  to  pursue.  We  identified  four  major  frames  (Good  Stranger,  Mission- 
oriented,  Rules  and  Procedures,  Authoritarian)  that  military  personnel  leverage  in  various 
situations.  Tasks  and  objectives  call  for  different  frames.  In  a  peacekeeping  environment,  one 
would  benefit  from  maintaining  a  Good  Stranger  frame  to  build  trust  with  a  community,  rather 
than  seeking  ways  to  manipulate  and  control  with  an  authoritarian  frame.  Team  leaders  would 
benefit  from  knowing  the  types  of  frames  their  team  members  are  partial  towards  and  which 
frames  they  dislike,  ignore,  or  do  not  possess.  Importantly,  this  information  is  most  valuable  to 
the  team  leader  before  they  start  their  mission  so  they  can  direct  personnel  to  the  most 
appropriate  tasks.  For  example,  if  someone  utilizes  the  rules  and  procedure  frame,  they  may  not 
be  best  suited  for  a  task  that  involves  creativity  and  thinking  on  their  feet.  Someone  apt  in  social 
interaction  skills  may  be  underutilized  working  on  equipment  and  not  interacting  with 
community  members. 

What  are  the  dominant  and  non-dominate  frames  that  this  group  uses  to  make  sense  of 
situations? 

•  The  most  preferred  frame  is  the  Good  Stranger  frame  (.73) 

•  The  least  preferred  frame  is  the  Authoritarian  frame  (.43) 

•  Frame  preference  scores  are  percentages  (total  average  frame  score/total  possible  frame 
score).  The  rankings  (1  =  least  prefer  -  4  =  most  prefer)  for  each  of  the  ten  questions 
were  summed  for  each  frame  (GS,  M,  R&P,  A).  This  raw  score  was  averaged  across  all 
participants.  The  average  frame  scores  were  then  divided  by  the  maximum  frame  score 
(40). 

This  sample  displayed  more  Good  Stranger  proclivities  than  we  expected.  Many  assume  the 
Good  Stranger  approach  is  out  of  sync  with  the  conventional  warfighter’s  psyche.  However,  this 
data  suggests  otherwise.  The  group,  on  average,  preferred  responding  with  actions  in  line  with  a 
Good  Stranger  frame.  They  favored  cooperation,  voluntary  compliance  and  rapport  building 
strategies  to  build  trust  with  civilians.  Our  findings  suggest  that  the  Good  Stranger  training 
strategies  being  developed  by  the  SSIM  teams  may  not  be  as  difficult  to  employ  as  many 
expected. 
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Not  surprisingly,  this  group  showed  an  almost  equally  strong  preference  for  the  Rules  and 
Procedures  frame  (.71).  They  highly  prioritized  actions  that  insisted  on  following  rules,  which  is 
characteristic  of  hierarchical  organizations  such  as  the  military.  However,  complex  and  turbulent 
situations  require  warfighters  to  think  outside  of  simple  rules  and  procedures  that  are  oftentimes 
futile  in  these  environments.  Because  of  the  group’s  strong  preference  for  a  rule -based  frame, 
they  may  not  manage  chaotic  situations  effectively.  Simulation  training  can  provide  extensive 
real-world  experience  in  these  complex  situations,  which  can  build  expertise  and  prepare 
warfighters  for  these  situations. 

Follow  on  research  should  investigate  if  and  how  warfighters  are  able  to  modify  their  frame 
depending  on  their  circumstances.  It  would  be  advantageous  for  training  personnel  to  know  if 
warfighters  can  switch  their  frame  quickly  as  situations  change  over  time.  Modem  warfare 
requires  the  military  to  engage  in  kinetic  and  peacekeeping  environments.  Oftentimes 
warfighters  must  simultaneously  juggle  both  types  of  situations  (e.g.  counter  insurgency 
missions).  More  work  is  needed  to  investigate  the  types  of  situations  warfighters  encounter  and 
the  optimal  frames  for  each  situation  type. 

Social  Skills  -  Preference  &  Confidence 

Among  the  list  of  SSIM  social  interaction  skills,  we  closely  examined  group  preference  and 
confidence  for  de-escalation,  voluntary  compliance,  rapport  building,  and  perspective  taking. 
Specifically,  we  examined  how  warfighters  prioritize  and  perceive  their  confidence  in 
performing  actions  that  manifest  these  abilities.  In  addition  to  these  soft  skills,  we  assessed 
preference  and  confidence  for  Security-oriented  actions.  Understanding  how  warfighters  balance 
soft  skills  with  security  can  have  important  training  implications. 

Which  skills  does  the  group  prefer  to  use? 

•  They  most  prefer  using  de-escalation  tactics  (.70),  in  comparison  to  the  other  skills 

•  They  also  prefer  to  use  security  (.60),  but  not  as  much  as  de-escalation  and  voluntary 
compliance  (.65). 

•  They  least  prefer  using  rapport  building  (.54),  in  comparison  to  the  other  skills 

•  Preference  scores  are  percentages  (total  average  preference  score/total  possible 
preference  score).  The  rankings  (1  =  least  prefer  -  4  =  most  prefer)  for  each  of  the  eleven 
scenario  questions  were  summed  for  each  learning  objective  (DE,  VC,  RB,  S).  This  raw 
score  was  averaged  across  all  participants.  The  average  preference  scores  were  then 
divided  by  the  maximum  frame  score  (44). 

Are  they  confident  using  these  skills? 

•  They  were  most  confident  in  providing  security  measures  in  all  scenarios  (.88). 

•  They  were  least  confident  building  rapport  (.75)  with  civilians  and  community  leaders 

•  Confidence  levels  are  percentages  (total  average  confidence  rating/total  possible 
confidence  ranking).  The  rankings  (1  =  Not  Confident  -  5  =  Very  Confident)  for  each  of 
the  eleven  questions  were  summed  for  each  frame  (DE,  VC,  RB,  S).  This  raw  score  was 
averaged  across  all  participants.  The  average  confidence  rankings  were  then  divided  by 
the  maximum  confidence  score  (25). 
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Among  the  soft  skills,  this  sample  preferred  response  options  that  characterized  de-escalation 
and  voluntary  compliance  tactics.  Interestingly,  they  showed  the  least  preference  and  confidence 
for  rapport  building,  which  includes  behaviors  such  as  searching  for  shared  goals,  showing 
empathy,  and  promoting  common  ground.  These  actions  load  onto  a  trust  building  (G.  Klein,  H. 
Klein,  Lande,  Borders,  Whitacre,  2015),  which  is  necessary  to  successfully  stabilize 
communities  in  unconventional  warzones.  Our  data  indicates  that  these  warfighters  are  not 
confident  in  their  rapport  building  skills;  therefore  they  may  not  be  adequately  trained  to 
effectively  perform  these  efforts.  Future  training  would  be  best  directed  to  emphasize  the 
importance  of  rapport  building  on  communities.  Although  these  skills  are  unconventional,  they 
can  be  critical  for  decreasing  hostility  and  maintaining  security  in  an  unstable  region.  Until 
warfighters  are  confident  in  their  skills,  they  will  not  be  likely  to  adopt  such  behaviors  in  the 
field.  Simulation  efforts  are  especially  valuable,  as  they  allow  trainees  build  experience  and 
confidence  developing  these  skills  in  realistic  situations. 

As  expected,  this  group  highly  prioritized  options  that  were  security-oriented.  Further,  this  group 
was  overwhelmingly  confident  performing  the  security  options.  Security-oriented  actions  are 
deeply  ingrained  in  military  culture  (rules  and  procedures),  which  provides  reasons  for  the 
sample’s  strong  tendencies.  The  ability  to  establish  and  maintain  security  throughout  a  mission  is 
critical  when  evaluating  success.  However,  the  means  of  obtaining  security  can  be  achieved  in 
various  ways.  Sometimes  it  may  be  prudent  to  de-escalate  a  situation  by  slightly  reducing 
security  measures  in  order  to  develop  long-term  trust  within  a  community.  The  social  skills 
outlined  by  the  SSIM  teams  provide  indirect  and  subtle  paths  to  achieve  security,  and  they 
achieve  this  without  vaunting  force  and  coercion.  Training  programs  should  leverage  the  positive 
long-term  impacts  that  strategic  social  interactions  (e.g.  rapport  building,  de-escalation)  have 
within  communities.  Law  enforcement  agencies  provide  great  examples  (e.g.  community 
policing)  for  the  military  to  learn  from  (Klein  et  al.,  2015).  As  warfighters  begin  trusting  the 
effectiveness  of  these  skills  (e.g.  seeing  results),  they  will  be  more  confident  in  adopting  and 
employing  them  in  the  field. 

Does  the  type  of  scenario  influence  the  skills  they  prefer  to  use? 

•  Tense  situations  (with  potential  to  become  kinetic) 

o  This  group  preferred  using  security  above  the  other  skills  in  some  stressful 

situations  (broadsided:  security  (.70)).  However,  sometimes  the  security-oriented 
approach  was  too  extreme.  For  example,  in  the  scenario,  disaster  relief,  the 
participants  were  opposed  to  the  security  approach  (.47),  to  shoot  the  hostiles, 
even  though  they  were  justified  to  do  so. 

•  Negotiating  situations 

o  The  group  preferred  using  de-escalation  tactics  when  they  had  to  manage  groups 
of  people  (Shots  Into  a  Crowd  (.80);  Weapons  Harvest  (.76)) 

The  scenarios  used  in  this  study  required  the  group  to  manage  a  range  of  experiences,  and  there 
were  no  right  or  wrong  response  types.  There  were  developing  trends  favoring  de-escalation 
options  for  scenarios  that  focused  on  negotiating  a  conflict.  De-escalation  includes  attempting  to 
reduce  intensity  of  situations,  remaining  calm  and  impartial,  and  thoroughly  explaining  one’s 
actions  in  a  rational  manner.  The  data  suggests  that  warfighters  can  effectively  modify  their 
response  type  in  fit  their  circumstances.  More  work  is  needed  to  investigate  how  skill 
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preferences  and  confidence  levels  change  in  response  to  various  situation  types. 

Skill  Competency 

The  last  section  of  Good  Stranger  Diagnostic  Tool  examined  competency  for  the  learning 
objectives  specified  in  the  previous  section.  More  specifically,  we  examined  how  accurately  the 
group  could  identify  behaviors  that  support  soft  skills.  Adaptive  training  solutions  would  greatly 
benefit  from  knowing  how  the  trainee  understands  the  target  skills.  More  resources  can  be 
directed  improving  areas  of  weakness  (e.g.  clarifying  misunderstandings),  and  less  time  can  be 
spent  in  areas  with  high  competency. 

Are  they  competent  in  using  the  identified  soft  skills? 

•  Asa  group,  they  are  most  competent  in  gaining  voluntary  compliance  (87%,  10.45 
correct/ 12  possible) 

•  Asa  group,  they  were  least  competent  in  taking  another’s  perspective  (50%,  6/12). 

•  The  group  was  fairly  competent  in  rapport  building  (80%)  and  de-escalation  (81%). 

The  group  showed  the  least  competence  for  the  skills  involved  in  taking  another  person  or 
group’s  perspective.  Perspective  taking  is  strongly  linked  to  empathy,  the  ability  to  recognize 
and  share  other’s  feelings,  which  is  a  critical  skill  for  building  trust.  It  is  possible  that  low 
competency  for  perspective  could  be  linked  to  an  inability  to  think  hypothetically.  Warfighters 
are  unable  to  visualize  themselves  in  another  person’s  situation;  therefore,  they  don’t  recognize 
the  importance  of  this  skill.  Training  program  should  target  perspective  taking  and  empathy 
skills,  so  warfighters  can  become  attuned  to  the  needs  and  goals  of  others. 
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The  objective  of  this  project  was  to  understand  why  and  how  some  police  officers  and 
military  personnel  are  more  effective  than  others  at  managing  civilian  encounters  without 
creating  hostility  -  ‘Good  Strangers’  (GSs).  We  conducted  cognitive  task  analysis  (CTA) 
interviews  with  1 7  US  police  officers  and  24  US  warfighters  (Marines  and  Army  soldiers). 

The  interviews  yielded  a  total  of  38  incidents  ( 1 7  police  and  2 1  military),  which  we  used  to 
identify  critical  skills  for  functioning  as  GSs.  These  skills  centred  on  having  a  sensemaking 
frame  that  established  a  professional  identity  as  a  GS  -  Someone  who  seeks  opportunities 
to  increase  civilian  trust  in  police/military.  This  frame  requires  skills  in  gaining  voluntary 
compliance,  building  rapport,  trading  off  security  and  other  frames  versus  trust  building, 
and  taking  the  perspective  of  civilians. 


Practitioner  points 

•  To  work  effectively  with  civilians,  police  and  military  personnel  need  to  use  a  Good  Stranger  frame, 
which  casts  each  encounter  as  an  opportunity  to  build  trust. 

•  This  GS  frame  requires  skills  such  as  trading  off  security  to  be  seen  as  trustworthy,  perspective  taking, 
gaining  rapport,  gaining  voluntary  compliance  rather  than  coercive  compliance,  and  de-escalating  tense 
situations. 

•  The  GS  frame  may  be  surprisingly  easy  to  acquire  for  some  police  and  military;  observation  of  role 
models  and  their  effectiveness  seems  to  be  a  powerful  training  opportunity.  Other  training  leverage 
points  involve  peer  pressure,  becoming  more  effective  at  gaining  civilian  cooperation,  and  recognizing 
the  problems  created  by  failing  to  build  trust. 


The  goal  of  this  project  was  to  understand  why  and  how  police  officers  and  military 
personnel  can  interact  with  civilians  to  gain  good  will  and  reduce  antagonism.  We  studied 
warfighters  and  police  officers  because  they  face  many  of  the  same  challenges  and  have 
learned  lessons  that  might  be  useful.  For  police,  civilian  encounters  are  a  primary  aspect  of 
the  job,  whereas  civilian  encounters  have  not  been  a  high  priority  for  the  military  until 
recently.  Police  officers  have  a  variety  of  responsibilities  in  maintaining  law  and  order 
(e.g. ,  arresting  criminals  and  providing  security  in  their  jurisdictions).  Police  are  trained  to 
gain  coercive  compliance  including  various  forms  of  force  and  direct  threats  of  firearms  or 


*Correspondence  should  be  addressed  to  Gary  Klein,  MacroCognition  LLC,  120  West  Second  Street,  Suite  1210,  Dayton,  OH 
45402-8522,  USA  (email:  gary@macrocognition.com). 
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tasers.  However,  some  officers  are  skilled  at  gaining  voluntary  compliance,  which  is  less 
likely  to  make  civilians  resentful  and  antagonistic. 

The  military  faces  similar  issues.  Historically,  the  US  military  trained  warfighters  to 
engage  against  traditional  national  forces.  They  were  prepared  in  the  strategies  and  tactics 
of  the  battlefield  that  involved  well-defined  adversaries  and  clear  rules  of  engagement. 
Soldiers  were  to  adopt  a  ‘warrior’  mindset.  However,  US  troops  are  increasingly  called 
upon  to  engage  paramilitary  forces  and  nationless  adversaries  in  unconventional  actions 
that  are  less  well  defined  and  predictable.  They  often  are  charged  with  fostering  the 
stabilization  of  local  communities.  To  succeed  in  these  tasks,  they  have  to  cultivate  good 
relations  with  citizens.  Simultaneously  managing  two  roles  -  fighting  adversaries  and 
supporting  civilians  -  presents  difficult  challenges.  Just  like  police,  warfighters  are  more 
effective  in  encounters  with  civilians  if  they  can  reduce  coercion  and  prevent  antagonism. 

But  how  can  police  and  military  personnel  obtain  voluntary  compliance?  We  recognize 
that  in  this  context,  compliance  is  never  truly  voluntary  because  it  is  backed  up  by  the 
potential  to  escalate  to  more  coercive  forms.  However,  this  type  of  voluntary’  compliance 
-  compliance  gained  without  direct  threats  and  overt  coercion  -  is  less  likely  to  make 
civilians  resentful  and  antagonistic  than  compliance  gained  through  physical  force,  overt 
threats,  and  outright  coercion. 

In  2011,  Defense  Advanced  Research  Programs  Agency  (DARPA)  initiated  a 
programme,  ‘Strategic  Social  Interaction  Modules’  (SSIM),  to  use  experiential  simulation 
and  other  techniques  to  teach  social  skills  so  that  military  personnel  can  consistently  gain 
voluntary  compliance  and  maintain  cooperative  working  arrangements  with  civilians. 
The  goal  of  the  SSIM  programme  is  to  help  military  personnel  develop  the  social 
interaction  skills  and  human  dynamics  proficiencies  demonstrated  by  those  who  are  Good 
Strangers  (GS)  -  To  prepare  them  to  create  positive  outcomes  in  social  encounters.  The 
nickname  for  the  SSIM  programme  is  the  ‘Good  Strangers’  project  because  the  intent  is  to 
transform  military  personnel  into  agents  who  elicit  trust  and  cooperation  rather  than 
hostility. 

There  is  a  large  literature  on  ways  for  police  and  military  to  accomplish  their  missions 
without  being  provocative.  We  surveyed  a  range  of  publications  on  using  ‘soft’  methods  of 
persuasion  and  influence  (Cialdini,  1993;  Glennon,  2010;  Thaler  &  Sunstein,  2008; 
Thompson  &Jenkins,  1993/2004;  Wilson,  201 1).  There  is  no  lack  of  speculation  about  the 
skills  needed  by  GSs,  but  we  did  not  find  any  in-depth  studies  of  how  these  skills  were 
formed  and  implemented  by  police  officers  and  warfighters.  Therefore,  we  conducted 
critical  decision  method  (CDM)  interviews  (Klein,  Calderwood,  &  MacGregor,  1989),  a 
form  of  cognitive  task  analysis,  with  police  and  military  personnel  who  were  identified  as 
GSs,  to  try  to  understand  what  enabled  them  to  work  well  with  civilians.  We  reviewed  4 1 
military  and  police  reports  and  identified  24  different  knowledge/skills/abilities 
mentioned  in  these  documents. 

Currently,  police  academies  include  some  form  of  preparation  for  interacting  with 
civilians  but  have  expressed  a  strong  interest  in  expanding  this  type  of  training.  The 
military  has  sought  to  provide  GS  training  as  a  result  of  the  counterinsurgency  efforts  in 
Iraq  and  Afghanistan,  but  the  training  is  extremely  expensive,  involving  a  large  number  of 
role  players  in  mock-up  villages  and  city  neighbourhoods.  With  the  cessation  of  US 
military  activities  in  Iraq  and  the  wind-down  in  Afghanistan,  and  the  reduced  interest  in 
counterinsurgency  operations,  these  training  facilities  are  being  shrunk  or  closed. 
Nevertheless,  military  forces  will  still  have  to  interact  with  civilians  and  to  act  as  GSs  even 
when  they  are  not  conducting  counterinsurgency  missions.  Therefore,  the  SSIM  GSs 
programme  should  continue  to  be  relevant  to  the  military  as  well  as  to  police  departments. 
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As  part  of  the  SSIM  programme,  we  conducted  cognitive  task  analysis  interviews  with 
police  and  military  to  gain  a  better  understanding  of  the  dynamics  of  GSs,  develop  training 
materials  and  programmes,  and  advise  other  SSIM  partners  who  are  developing  advanced 
experiential  simulations.  This  article  describes  our  initial  findings  about  how  police 
officers  and  military  personnel  GSs  make  sense  of  challenging  civilian  encounters. 


Method 

Participants 

The  participants  were  17  experienced  police  officers  from  four  jurisdictions  within  the 
United  States  and  24  warfighters,  including  both  Marines  and  Army  soldiers.  The  mean  age 
was  39- 7  years.  Three  of  the  police  participants  were  female,  and  all  of  the  military 
participants  were  male. 

Supervisors  from  each  organization  selected  the  participants.  We  requested  interview 
time  with  police  and  military  personnel  who  were  acknowledged  by  their  supervisors  to 
be  GS  exemplars  -  Professionals  who  had  demonstrated  superior  abilities  (compared  to 
their  peers)  in  engaging  with  civilians  and  in  de-escalating  rather  than  escalating  situations 
involving  conflict.  The  objective  of  the  data  collection  was  to  identify  the  skills  used  by 
GSs  and  to  understand  how  GSs  make  sense  of  challenging  civilian  encounters. 


Data  collection  method 

Job  shadowing 

The  interviewers  conducted  four  ride-alongs:  8  to  10  hr  shifts  riding  with  four  police 
officers  as  they  conducted  their  patrols.  The  job  shadowing  involved  one  observer  and 
one  police  officer  in  a  patrol  car  during  a  shift. 


COM  interviews 

We  used  the  CDM  as  our  CTA  approach  (Crandall,  Klein,  &  Hoffman,  2006;  Klein  et  al., 
1989).  The  CDM  is  an  interview-based  knowledge  elicitation  technique  that  elicits  critical 
incidents  to  expose  different  types  of  expertise.  The  rationale  is  that  expertise  becomes 
important  in  handling  tough  cases.  The  CDM  is  a  qualitative  method,  intended  to 
complement  more  quantitative  data  collection  efforts  used  by  other  research  teams  within 
the  DARPA  SSIM  programme.  Two  interviewers  conducted  all  41  CDM  interviews,  usually 
as  a  team  except  for  four  cases  where,  for  scheduling  reasons,  they  had  to  conduct  solo 
interviews  in  parallel.  Both  interviewers  have  many  decades  of  experience  conducting 
CDM  interviews.  The  interview  data  were  collected  in  office  spaces  at  the  participants’ 
work  settings. 

Each  CDM  interview  consisted  of  four  sweeps  through  every  incident:  A  brief  initial 
description,  a  timeline  for  the  entire  incident,  identification  of  decisions  and  of  changes  in 
situation  awareness  during  the  incident,  and  final  probes  (e.g.,  hypothetical  variations). 
We  asked  about  specific  incidents  that  the  interviewee  found  challenging.  The  interviews 
with  police  examined  challenging  encounters  with  civilians.  The  interviews  with  military 
personnel  examined  critical  incidents  the  interviewee  had  experienced  with  civilians  and 
with  local  military  personnel  during  overseas  deployments.  The  interviews  lasted  39- 
123  min.  The  interviews  were  voice  recorded  and  yielded  more  than  1,000  pages  of 
transcripts. 
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Most  interviews  explored  one  or  two  incidents,  but  several  had  as  many  as  three  or  four 
incidents.  Three  participants  lacked  any  incident  with  sufficient  detail  to  allow  scoring. 
We  excluded  one  story  that  came  prior  to  a  participant  s  military  experience  as  well  as  six 
undeveloped  stories. 

The  41  interviews  yielded  92  incidents  (44  police,  48  military).  We  recorded  the 
specific  features  of  each  incident  along  with  the  initiating  event  and  the  interviewee’s 
early  sense  of  the  situation,  including  goals,  threats,  and  key  participants.  We  also 
recorded  the  decision  points  the  interviewee  encountered  in  each  incident.  We  defined 
decision  points  as  critical  moments  in  the  incident  requiring  the  interviewee  to  make  a 
decision.  The  police  incidents  included  domestic  violence  calls,  patrolling  a  gang  funeral, 
and  traffic  stops.  The  military  incidents  involved  a  variety  of  missions,  such  as  dispersal  of 
protestors,  transporting  supplies,  providing  bank  protection  during  money  deliveries, 
de-escalating  community  anger  at  an  accidental  shooting,  and  handling  check  points.  The 
material  that  follows  includes  examples  from  our  interviews.  The  details  of  the  examples 
have  been  altered  to  guard  the  anonymity  of  participants  while  retaining  the  dynamics  of 
the  interaction. 


Data  coding 

Interviewer  global  Good  Stranger  ratings 

Both  interviewers  independently  reviewed  each  interview  transcript  and  rated  the 
interviewee  on  a  7-point  scale,  where  7  =  good  stranger  and  1  =  bad  stranger.  These 
ratings  were  based  on  a  global  impression  from  the  interviews  and  the  way  the  incidents 
were  handled.  Intraclass  correlations  (ICC)  were  used  to  evaluate  rater  consistency 
(Shrout  &  Fleiss,  1979).  The  interviewers  showed  significant  agreement,  ICC 
(3,2)  =  .944,  in  their  ratings.  Subsequent  to  the  ratings,  to  justify  their  global 
impressions,  the  interviewers  tried  to  identify  the  criteria  they  were  using.  The 
following  characteristics  contributed  to  their  GS  ratings:  Showing  genuine  concern  for 
the  needs  of  civilians,  wanting  to  make  a  difference  in  people’s  lives,  anger  at  other 
police  officers  acting  inappropriately  (e.g.,  abusing  their  authority),  refusal  to  take 
provocations  and  insults  personally  (e.g.,  perspective  taking  and  emotion  regulation), 
taking  pleasure  in  getting  thanked  after  giving  citations,  and  wanting  to  stay  calm  in  tense 
situations. 


Incident  and  decision  point  coding 

To  better  understand  the  behavioural  and  cognitive  strategies  involved  in  being  a  GS,  we 
selected  each  participant’s  most  reflective  incident  for  more  thorough  analyses.  A 
participant’s  most  reflective  incident  had  to  be  a  complete  incident  in  which  the 
interviewee  was  heavily  involved  in  the  decision-making  and  outcome  of  the  situation.  If 
the  interviewee  provided  multiple  incidents  meeting  these  criteria,  we  selected  the 
incident  that  contained  the  most  decision  points. 

Two  independent  raters,  not  involved  in  the  interview  process,  independently 
analysed  the  selected  38  critical  incidents  (police:  17  incidents,  military:  21  incidents). 
Three  of  the  41  interviews  did  not  generate  an  incident  that  could  be  coded.  Prior  to 
coding,  both  raters  attained  mutual  understanding  of  the  working  definitions  of  the  GS 
cognitive  and  behavioural  dimensions  described  below. 
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Coding  scheme 

The  interviewers’  global  impressions  and  additional  research  conducted  within  the 
DARPA  SSIM  programme  were  then  used  to  develop  a  behavioural  and  cognitive  model  of 
GS  sensemaking.  To  mitigate  overlapping  features,  we  identified  the  core  cognitive  and 
behavioural  principles  involved  in  comprising  the  GS  mindset  (Tables  1  and  2).  Due  to  the 
nature  of  our  data,  we  differentiated  behaviours  from  cognitive  processes. 


Behavioural  strategies 

These  strategies  involved  six  actions:  De-escalates  tense  situations,  builds  rapport,  reads 
non-verbal  cues,  takes  prudent  risks,  shows  curiosity  about  the  civilian’s  odd  behaviours, 
and  polices  his/her  own  colleagues  who  may  be  acting  inappropriately.  These  behaviours 
could  be  easily  identified  within  the  incidents.  Raters  coded  actions  taken  by  the 
interviewee.  For  every  decision  point  within  an  incident,  the  six  behavioural  strategies 
were  coded  as  —  1  =  maladaptive,  1  =  adaptive,  or  0  =  not  present  (some  incidents  did 
not  contain  relevant  context  to  each  strategy).  Cohen’s  kappa  was  used  to  evaluate  rater 
consistency  (Table  1). 


Cognitive  strategies 

We  identified  five  frames  that  comprise  that  GS  mindset:  Building  trust  in  civilians, 
perspective  taking,  long  horizon  for  anticipating  consequences  of  actions,  voluntary 
compliance,  and  self-control.  Coders  inferred  the  cognitive  strategies  from  transcript  data 
as  holistic  beliefs  and  ideologies.  Unlike  the  GS  behavioural  coding  scheme,  where 
strategies  were  coded  for  each  decision  point  within  a  selected  incident,  cognitive 
strategy  ratings  were  based  on  global  impressions  from  the  entire  incident.  Although 
frame  types  (i.e.,  cognitive  strategies)  and  actions  (i.e.,  behavioural  strategies)  were 
aligned,  there  were  multiple  incidents  in  which  a  participant’s  actions  did  not  reflect  his/ 
her  frame  type.  Each  cognitive  strategy  was  scored  using  a  7-point  Likert  scale 
(— 3  =  strong  avoidance  to  3  =  strong  preference)  with  zero  indicating  no  preference 


Table  I.  List  of  SSIM  Good  Stranger  behavioural  strategies,  descriptions  and  inter-rater  reliability 
agreement 


Strategy 

Description 

Kappa 

De-escalates 

Attempts  to  reduce  the  intensity  of  a  situation  or  conflict;  remains  calm 
and  is  able  to  explain  actions  rationally 

52** 

Building  Rapport 

Seeks  to  develop  a  positive  relationship  with  others  using  an  inviting 
approach 

.67** 

Reading  Non-verbal 

Reads  body  language  and  facial  cues  to  help  make  situational 
assessments 

.85** 

Prudent  Risk 

Understands  importance  of  security  but  is  able  to  gain  cooperation 
without  provoking  antagonism  (e.g.,  trading  off  security  for  trust) 

5 1** 

Curiosity/Ground 

Truth 

Seeks  to  understand  the  heart  of  the  problem;  impartial  and  lets  the 
evidence  direct  blame 

.46** 

Policing  Own 

Able  to  manage  team  members,  holding  everyone  accountable. 
Addresses  problems  when  they  arise  and  admits  personal  faults 

57** 

Note.  **Significant  at  the  .001  level. 
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Table  2.  List  of  SSIM  cognitive  strategies,  descriptions  and  intraclass  correlations  (rater  agreement) 


Strategy 

Description 

ICC 

Building  Trust 

Promotes  positive  encounters,  uses  the  situation  to  gain 
the  faith  of  civilians.  Displays  Meyers  et  al.  1995  dimension 
of  trust:  Ability,  benevolence,  and  integrity 

74** 

Perspective  Taking 

Understands  other’s  actions  through  ability  to  see  a  situation 
from  another  perspective 

.85** 

Long-Term  Mindset 

Understands  long-term  consequences  for  their  actions; 
considers  long-term  implications  before  taking  certain  actions 

.65** 

Voluntary  Compliance 

Shows  willingness  to  employ  verbal  and  non-verbal  strategies 
to  gain  voluntary  rather  than  coercive  compliance 

37** 

Self-control 

Careful  not  to  project  arrogance  or  dominance  in  a  negative 
way;  does  not  take  provocations  personally.  Able  to  control 
own  body  language 

75** 

Notes.  ICC,  Intraclass  correlation  coefficient;  **Significant  at  .001  level. 


for  the  strategy.  A  7-point  scale  was  used  because  the  cognitive  strategies  were  more 
subjective  than  behavioural  strategies.  With  this  scale,  we  hoped  to  gauge  the  extent  to 
which  the  participant  demonstrated  each  cognitive  strategy.  We  would  be  unable  to  do 
this  using  the  ‘yes/no’  scale  employed  for  the  behavioural  strategies.  ICC  were  used  to 
evaluate  rater  consistency  (Table  2). 


Results 

The  interviewers’  GS  ratings  were  averaged  for  each  participant,  generating  individual  GS 
scores.  The  GS  ratings  for  police  (M  =  5.09,  SD  =  1.73)  and  military  (M  =  5.23, 
SD  =  1.42)  participants  did  not  differ,  f(35)  =  —0.26,  p  >  .05.  Despite  our  request  for 
interviews  with  professionals  who  were  all  GSs,  our  ratings  showed  that  some  stood  out 
from  the  others. 

This  was  an  inductive,  exploratory  study  of  the  qualities  and  capabilities  of  GSs. 
Therefore,  we  were  not  testing  a  specific  model  or  hypothesis.  A  correlation  matrix  was 
used  to  investigate  individual  relationships  between  behavioural  and  cognitive  GS 
strategies  (Table  3).  De-escalation,  building  rapport,  prudent  risk,  perspective  taking, 
long  horizon,  voluntary  compliance,  and  self-control  were  significantly  related  to 
participant’s  GS  ratings.  Conversely,  the  participants’  ability  to  read  non-verbal 
behaviours,  curiosity,  and  policing  own  did  not  relate  to  the  GS  ratings.  Most  notably, 
we  found  that  building  trust  was  strongly  related  to  participants’  GS  ratings,  r(35)  =  .69, 
p  <  .001. 

All  but  three  behavioural  GS  strategies  (non-verbal,  curiosity,  and  policing  own)  were 
significantly  related  to  building  trust.  All  cognitive  GS  strategies  were  significantly  related 
to  building  trust.  Among  these  four  cognitive  strategies,  perspective  taking  was  the  most 
strongly  related,  r(35)  =  .84, p  <  .00. 


Behavioural  Good  Stranger  strategies  model 

A  multiple  regression  analysis  was  used  to  examine  whether  the  GS  behavioural  strategies 
significantly  predicted  the  participants’  GS  rating  (Table  4).  This  model,  containing  six 
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Table  4.  Summary  of  multiple 

regression  for  behavioural  predictor  variables  to  Good  Stranger  rating 

Predictor  variable 

8 

SE 

P 

t 

P 

De-escalation 

1.22 

0.86 

.25 

1.43 

.16 

Building  Rapport 

1.26 

0.67 

.33 

1.87 

.07 

Non-verbal 

2.1  1 

1.07 

-.28 

-1.97 

.06 

Prudent  Risk 

1.98 

1.14 

.25 

1.73 

.09 

Curiosity 

0.24 

1.31 

-.03 

-0.19 

.85 

Policing  Own 

0.71 

0.88 

.13 

0.82 

.42 

Note.  B,  un-standardized  beta  coefficient;  SE,  standard  error; 

|3,  standardized  beta  coefficient;  t, 

t-test 

statistic;  P,  significance  value. 

Table  5.  Summary  of  multiple 

regression  for  cognitive  predictor  variables  to  Good  Stranger  rating 

Predictor  variable 

8 

SE 

P 

t 

P 

Building  Trust 

0.70 

0.25 

.71 

2.73 

.01** 

Perspective  Taking 

-0.01 

0.26 

-.1  1 

-0.40 

.69 

Long  Horizon 

0.05 

0.17 

.06 

0.28 

.78 

Voluntary  Compliance 

-0.25 

0.21 

-.25 

-1.17 

.25 

Self-control 

0.24 

0.20 

.27 

1.22 

.23 

Notes.  8,  un-standardized  beta  coefficient;  SE,  standard  error;  (3,  standardized  beta  coefficient;  t,  t-test 
statistic;  P,  significance  value;  **Significant  at  the  .0 1  level. 

predictors,  explained  48%  of  the  variance,  R 2  =  .48,  F(6,  30)  =  4.66,  p  <  .01. 
Interestingly,  none  of  the  individual  GS  behavioural  strategies  significantly  predicted  GS 
rating.  Two  of  the  behaviours  achieved  a  marginal  level  of  significance,  building  rapport, 
(3  =  .33,  f(30)  =  1.87,/)  =  .07,  and  prudent  risk,  (3  =  .25,  7(30)  =  1.73,/)  =  .09. 


Cognitive  Good  Stranger  strategies  model 

A  multiple  regression  analysis  was  also  used  to  test  whether  the  cognitive  GS  strategies 
significantly  predicted  participants’  GS  ratings  (Table  5).  The  result  of  this  regression 
indicated  that  the  five  predictors  explained  51%  of  the  variance,/?2  =  .51,  F(5 ,31)  =  6.39, 
p  <  .001.  This  model  revealed  that  building  trust  was  a  highly  significant  predictor  of  GS 
ratings,  (3  =  .71,  7(31)  =  2.73 ,p  =  .01. 

An  additional  multiple  regression  model  was  employed  to  identify  behavioural  and/or 
cognitive  GS  factors  that  loaded  onto  building  trust.  This  model  significantly  predicted 
building  trust,  explaining  86%  of  the  variance,  R2  =  .86,  /;(10,  26)  =  15.44,  p  <  .001, 
matching  the  finding  from  the  correlational  data  presented  in  Table  3.  Two  behavioural 
GS  strategies,  perspective  taking  and  prudent  risk,  significantly  predicted  building  trust, 
(3  =  .47,  7( 25)  =  2.94,/)  =  .01  and  (3  =  .22,  7( 25)  =  2.56,  p  =  .01  respectively. 

Discussion 

The  purpose  of  this  project  was  to  better  understand  what  it  means  to  be  a  ‘Good  Stranger’ 
in  encounters  with  civilians.  Our  study  suggested  multiple  factors  are  significantly  related 
to  participant’s  GS  ratings  (Table  3).  Among  all  cognitive  and  behavioural  GS  strategies, 
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building  trust  stood  out  as  the  most  significant  predictor  variable  of  participant’s  GS 
ratings.  Multiple  regression  analysis  revealed  that  the  ability  to  take  another’s  perspective 
and  gauge  prudent  risk  significantly  predicted  building  trust.  There  was  a  temptation  to 
formulate  a  GS  model  around  these  factors,  given  that  they  have  some  empirical  support. 
However,  we  chose  a  different  approach.  We  did  not  want  to  compile  a  laundry  list  of  GS 
features  -  A  list  that  could  be  expanded  in  the  future  but  lacked  coherence. 

Instead,  we  used  a  sensemaking  perspective  to  understand  GSs.  We  reviewed  our 
interview  materials  to  try  to  capture  how  GSs  make  sense  of  civilian  encounters  in  ways 
that  the  participants  with  lower  GS  ratings  did  not.  We  used  the  data/frame  model  of 
sensemaking  (Klein,  Moon,  &  Hoffman,  2006a, b)  to  guide  our  approach.  We  did  not  apply 
all  or  even  most  of  the  aspects  of  the  data/frame  model  to  our  findings.  We  used  the 
aspects  that  we  believed  would  help  us  gain  a  better  understanding  of  GSs.  We  posited  a 
GS  frame  based  on  our  CTA  interviews.  The  concept  of  a  GS  frame  was  not  hypothesized  at 
the  outset  of  our  research.  Our  interviews  suggested  a  GS  frame  of  seeking  to  build  trust 
and  that  this  trust-building  frame  could  be  considered  as  a  form  of  sensemaking. 
Warfighters  and  police  officers  with  this  frame  seem  to  interpret  situations  differently  than 
those  who  lack  the  GS  frame  and  rely  on  the  standard  frames  of  self-protection,  mission 
accomplishment,  and  compliance  with  regulations  to  sort  out  what  is  important  in  a 
situation. 

We  then  augmented  this  account  by  drawing  on  the  recognition-primed  decision 
(RPD)  model  (Klein,  1998)  which  postulates  that  recognizing  a  situation  involves  four 
aspects:  Cues  to  monitor,  expectancies,  plausible  goals,  and  potential  courses  of  action. 
We  used  these  four  aspects  as  four  slots  in  a  sensemaking  frame.  In  so  doing,  we  believe  we 
have  made  a  theoretical  contribution  by  synthesizing  aspects  of  the  RPD  model  with  the 
data/frame  model.  Thus,  police  and  military  personnel  with  a  GS  frame  will  notice 
different  cues,  form  different  expectancies,  generate  different  goals,  and  consider 
different  courses  of  action  than  those  who  lack  or  do  not  use  a  GS  frame. 


Good  Stranger  frame  -  Gaining  civilian  trust 

We  speculate  that  the  basic  GS  frame  is  about  trying  to  gain  the  trust  of  civilians.  Figure  1 
shows  trust  building  as  the  core  GS  frame.  The  slots  in  this  frame  identify  what  a  GS  would 
expect  in  the  process  of  building  trust,  the  cues  a  GS  would  be  sensitized  to,  the  goals  a  GS 
would  have  relative  to  building  trust,  and  the  kinds  of  actions  a  GS  would  consider  in  an 
encounter.  The  GS  frame  would  involve  different  expectancies,  cues,  goals,  and  actions 
than  alternative  frames  such  as  trying  to  ensure  security,  accomplishing  a  mission, 
following  regulations,  or  establishing  dominance  over  civilians.  Figure  1  shows  these 
other  frames;  the  GS  frame  co-exists  with  the  others.  The  frames  for  security,  mission 
accomplishment,  and  adherence  to  regulations  are  essential  for  police  and  military 
personnel;  it  is  the  GS  frame  that  is  optional.  Many  police  and  military  personnel  do  not 
have  or  use  a  frame  for  building  trust  in  their  encounters  with  civilians. 

Trust  appears  to  be  critical  for  establishing  and  maintaining  the  good  will  of  civilians 
and  even  adversaries.  Mayer  and  his  colleagues  (Mayer,  Davis,  &  Schoorman,  1995; 
Schoorman,  Mayer,  &  Davis,  2007),  working  primarily  with  Westerners  in  organizational 
settings,  identified  ability,  benevolence,  and  integrity  as  dimensions  critical  for  establish¬ 
ing  trust  during  encounters.  Their  research  suggests  that  trust  can  reduce  hostility, 
increase  information  flow,  and  garner  co-operation.  Trust  eases  negotiation  and  increases 
operational  effectiveness  during  complex  and  dynamic  interactions.  Gottman  (201 1)  has 
investigated  the  centrality  of  trust  for  relationships  between  couples.  The  dimensions 
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Figure  I .  The  Good  Stranger  frame  of  building  trust. 


used  to  assess  trustworthiness  vary  significantly  by  national  group  and  situation  (H.A. 
Klein  et  al.,  2014).  Dimensions  beyond  Mayer’s  three  include  cognition  (Markus  & 
Kitayama,  1991),  social  values,  and  priorities  (Hofstede,  1980)  as  well  as  the  ones 
identified  by  H.A.  Klein  et  al.  (2014):  Affect,  non-verbal  communication,  dialectical 
reasoning,  interdependence,  and  status.  Drawing  on  a  variety  of  literatures  and  research, 
but  primarily  from  the  CDM  data  we  collected  in  this  effort,  we  define  GSs  as  police 
officers  or  warfighters  who  seek  to  increase  trust  during  encounters  with  the  local 
populace  while  carrying  out  their  responsibilities. 

The  GS  frame  is  about  wanting  to  increase  trust  from  beginning  to  end  of  an  encounter, 
and  boosting  the  trust  given  to  the  organization  (i.e.,  other  police  or  military),  not  just  the 
individual.  The  GS  activity  seeks  to  build  social  capital.  In  contrast,  the  interviewees  with 
low  GS  scores  showed  little  evidence  that  they  viewed  encounters  with  civilians  as 
opportunities  to  build  trust. 

One  question  to  consider  is  whether  military  personnel  are  operating  under  the  same 
constraints  as  police.  Warfighters  need  to  gain  immediate  compliance  and  are  less 
concerned  with  establishing  relationships  because  they  rotate  fairly  often  in  contrast  to 
police  who  usually  work  within  the  same  community.  Therefore,  police  may  worry  more 
about  trusting  relationships  than  military  personnel.  However,  when  we  asked  our 
military  participants  whether  it  was  important  to  build  trusting  relationships,  they  argued 
that  while  they  themselves  might  never  see  a  civilian  again  after  an  encounter,  other 
warfighters  would  bear  the  consequences  of  their  interaction.  Thus,  GSs  in  both  police 
and  military  saw  value  in  short-term  and  long-term  perspectives. 

Building  trust  has  a  short-term  component  -  Viewing  any  encounter  as  a  means  of 
gaining  more  trust  for  oneself  and  one’s  service.  From  the  start  to  the  end  of  the  encounter, 
a  GS  is  trying  to  ‘move  the  needle’  and  be  seen  as  more  trustworthy,  which  will  bear 
dividends  in  the  short  term,  such  as  gaining  immediate  compliance,  and  the  long  term, 
such  as  increasing  cooperation  with  warfighters  who  enter  into  the  next  encounter. 

Building  trust  also  has  a  long-term  component,  which  is  to  engage  in  actions  and 
policies  that  encourage  civilians  to  take  actions  that  have  some  risk,  such  as  reporting  on 
improvised  explosive  devices  (IEDs),  providing  information  about  insurgents,  asking  for 
help,  providing  help,  and  so  forth.  An  example  of  the  long-term  trust  created  by  GSs  is  the 
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incident  we  heard  of  the  Scorpions  in  Iraq  -  The  nickname  of  an  Army  National  Guard  unit 
from  Idaho.  The  Iraqi  populace  was  struck  by  the  benevolence  and  integrity  of  the 
Scorpions,  especially  in  comparison  with  the  unit  they  replaced.  The  preceding  unit  had 
shown  little  regard  for  trust  building.  Instead,  they  sought  to  use  intimidation  to  gain 
compliance,  and  the  civilians  resented  this  treatment.  After  the  Idaho  National  Guard  unit 
rotated  in,  they  had  more  damage  to  undo.  This  unit  took  a  different  approach.  Despite 
maintaining  a  firm  defensive  posture  at  all  times,  they  strived  to  be  polite  and  courteous  to 
the  community  members.  They  respected  the  local  populace  and  recognized  they  were 
visitors  on  their  land.  To  work  effectively,  they  explained  their  actions  and  demonstrated 
transparency  when  possible.  This  unit  worked  hard  to  show  the  Iraqi  civilians  that  they 
were  trustworthy,  and  their  efforts  paid  off.  When  Iraqis  saw  vehicles  with  a  Scorpion 
logo,  they  acted  differently  and  were  more  cooperative  and  trusting.  Soon  the  Scorpions 
were  labelling  all  of  their  vehicles  with  a  Scorpion  sign  (sort  of  like  gang  graffiti)  and,  more 
importantly,  were  united  in  their  desire  not  to  spoil  their  ‘brand’. 

While  our  sensemaking  account  of  GSs  centres  on  their  trust-seeking  frame,  in  other 
contexts,  we  might  refer  to  a  mindset  that  they  have,  or  a  stance  they  take.  All  of  these 
terms  describe  what  sets  them  apart  -  At  least  in  our  interpretation  of  the  interview 
records.  We  primarily  use  frame  in  this  paper  because  it  fits  with  our  views  on 
sensemaking. 


Acquiring  the  Good  Stranger  frame 

The  interviewees  showed  great  variety  in  how  they  became  GSs.  Early  family  experience 
was  important  for  some  people.  One  military  interviewee  reported:  ‘It’s  probably  from  my 
family.  At  dinner,  you  could  have  an  argument  but  you  always  listened  to  what  the  others 
said.  That  way  everyone  come  out  getting  along.  Now  when  I  walk  into  a  meeting  I’m 
prepared  for  arguments  but  I  try  to  listen  to  others  needs  and  come  up  with  plans  that 
make  my  actions  more  acceptable  to  people  .  Another  interviewee  said:  ‘For  this  mission,  I 
went  back  to  how  I  was  raised:  To  greet  people,  to  smile  at  people,  to  shake  hands, 
because  that’s  what  I  knew  and  what  I’m  comfortable  doing’. 

Some  interviewees  adopted  a  GS  frame  as  a  rejection  of  negative  experiences.  One 
military  NCO  (non-commissioned  officer)  explained  that  ‘when  I  just  started  there  was  a 
lot  of  issues  with  hazing.  Things  happened  that  I  didn’t  particularly  care  for.  I  decided  I 
was  going  to  do  the  opposite  of  what  some  of  the  people  I  encountered  did’ . 

Prior  work  experience  also  shaped  interaction  with  citizens.  Military  interviewees 
with  experience  as  police,  social  workers,  or  missionaries  (several  of  the  National  Guard 
participants  were  Mormons)  would  mention  these  past  experiences.  One  officer 
described  a  less  usual  experience:  ‘One  summer,  I  was  a  security  guard  at  a  shopping 
mall.  People  were  there  to  have  fun.  But  people  can  get  angry  if  somebody  pushes  them 
and  it’s  crowded  and  hot.  One  of  the  things  I  learned  was  how  to  defuse  the  tensions 
before  you  have  a  situation.  I  used  this  defuse  the  crowd  concept’  overseas.  I’m  good  with 
groups  where  not  everybody  gets  along  as  long  as  they  have  a  common  goal’. 

Finally,  some  interviewees  were  driven  to  be  more  effective.  They  adopted  the  GS 
frame  because  it  helped  them  accomplish  their  missions.  One  reported:  ‘To  get  anywhere 
with  Afghanistan  or  Iraqi  nationals  the  nicer  you  are  to  them,  the  nicer  they  are  to  you’. 

Particularly  for  the  police,  a  GS  frame  can  be  an  important  aspect  of  professional 
identity.  For  example,  a  number  of  police  officers  described  how  they  started  out 
expecting  that  the  job  of  the  police  force  was  about  catching  and  arresting  criminals  and 
drawing  on  their  authority  to  gain  compliance.  But  somewhere  along  the  way,  often  with 
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an  experienced  mentor  -  for  example  a  field  training  officer  -  they  observed  a  different 
approach.  They  encountered  role  models  who  spoke  softly  rather  than  yelling,  who 
treated  civilians  with  genuine  respect,  and  as  a  result  were  extremely  effective.  One 
officer  explained  that  after  working  with  such  a  mentor,  he  cut  the  number  of  fights  and 
violent  encounters  by  90%,  using  methods  he  had  learned  for  gaining  voluntary 
compliance. 

Our  interviews  suggest  that  some  professionals  may  never  acquire  this  GS  frame.  It  is 
not  part  of  their  conceptual  repertoire.  Others  may  have  acquired  the  frame  but  do  not 
activate  it  very  often.  These  were  the  participants  to  whom  we  gave  low  GS  ratings. 


Training  police  and  military  to  be  Good  Strangers 

We  believe  that  it  may  be  possible  in  a  relatively  short  period  of  time  to  convince  military 
(and  police)  of  the  value  of  having  a  GS  frame  and  to  start  them  on  their  journey  to 
acquiring  the  skills  to  put  this  frame  into  action.  It  is  like  making  a  discovery,  although  the 
discovery  will  have  to  be  followed  by  practice  and  refinement  so  that  the  GS  frame 
becomes  more  natural. 

The  reason  we  think  that  the  acquisition  of  a  GS  frame  can  happen  relatively  quickly  is 
that  the  interviewees,  particularly,  the  police,  offered  many  examples  of  how  they  started 
off  without  any  appreciation  of  the  GS  frame  and  then  were  struck  by  seeing  it  in  action, 
used  effectively  by  others  whom  they  respected.  We  identified  several  different  pathways 
contributed  to  the  acquisition  of  a  GS  frame: 

•  Role  models, 

•  Peer  pressure, 

•  Increased  effectiveness, 

•  Norms  and  performance  expectations,  and 

•  Increased  security  and  consequences. 

The  most  dramatic  incidents  revolved  around  role  models.  The  police  in  particular 
described  how  watching  a  mentor,  perhaps  a  first  field  training  officer,  opened  their  mind 
and  changed  their  professional  identity.  Watching  someone  speak  softly  and  respectfully 
and  gain  compliance  illustrated  a  power  they  wanted  to  have.  We  did  not  obtain 
comparable  examples  of  GS  role  model  inspiration  in  the  interviews  with  military 
personnel.  Even  when  asked  about  potential  role  models,  none  of  the  military  interviews 
could  think  of  anyone. 

Peer  pressure  was  another  motivation.  Police  officers  do  not  want  to  ride  with  bad 
strangers  because  they  increase  risks  that  simple  conflicts  will  escalate  unnecessary. 
Military  personnel  seen  as  creating  unnecessary  hostility  were  sometimes  sidelined  and 
prevented  from  engaging  with  civilians,  even  though  their  courage  and  combat  skills  were 
appreciated.  The  pressure  was  primarily  negative,  not  to  behave  stupidly  and  disrespect¬ 
fully  to  civilians,  but  there  was  also  a  positive  component:  To  skilfully  elicit  voluntary 
compliance  where  possible. 

Several  participants,  both  military  and  police,  explained  that  they  wanted  to  learn  to  be 
GSs  because  of  increased  effectiveness.  In  many  situations,  GSs  can  accomplish  their 
mission  more  quickly  and  effectively,  and  with  better  consequences. 

Norms  and  performance  expectations  came  up  in  some  of  the  interviews.  In  some 
settings,  the  lesson  conveyed  was  that  this  is  what  other  military/police  are  able  to  do. 
In  these  situations,  the  leaders  expected  subordinates  to  act  as  GSs. 
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Increased  force  security  was  a  surprising  item,  especially  in  the  military.  Several 
participants  explained  how  long-term  security  is  enhanced  if  the  unit  is  perceived  as 
trustworthy,  as  in  the  example  of  the  Scorpions.  This  factor  ties  into  the  finding  in  the 
Results  section  about  the  importance  of  long-term  consequences. 

The  law  enforcement  community  already  has  some  training  programmes  for  the 
supporting  skills  shown  in  Figure  1,  that  is  building  rapport  and  resolving  conflicts.  The 
verbal  judo  approach  to  gaining  voluntary  compliance  (Thompson  &  Jenkins,  1993/2004) 
is  widely  used  throughout  the  United  States.  The  military  is  just  starting  to  stand  up 
training  in  these  GS  subskills.  As  stated  earlier,  we  identified  41  police  and  military 
documents  addressing  different  aspects  of  GS  behaviour.  However,  most  of  these 
documents  simply  acknowledged  the  importance  of  the  skill,  rather  than  setting  forth  a 
training  programme.  Furthermore,  none  of  the  material  we  reviewed  took  a  sensemaking 
perspective  to  GS  behaviours  or  conceptualized  a  GS  frame  of  seeking  to  build  trust  from 
the  beginning  to  the  end  of  each  encounter  with  civilians. 


Skills  for  supporting  the  Good  Stranger  frame 

The  behaviours  we  identified  in  Tables  3-5  informed  the  GS  diagram  shown  in  Figure  1, 
which  shows  a  set  of  other  frames  the  GSs  may  be  using  in  conjunction  with  the  GS  frame. 
Figure  1  also  refers  to  tactics  and  skills  that  are  needed  to  augment  trust-building  efforts; 
these  emerged  from  our  research. 

We  also  heard  several  examples  of  deliberate  attempts  at  ‘swift  trust’,  a  skill  at  rapidly 
gaining  trust  in  encounters  with  strangers  (Meyerson,  Weick,  &  Kramer,  1996), 
particularly  in  the  police  interviews. 


Coordinating  the  Good  Stranger  frame  with  active  sensemaking  frames 

Police  officers  and  warfighters  have  several  ways  for  making  sense  of  situations, 
particularly  maintaining  control  during  encounters  (Alpert  &  Dunham,  2004),  ensuring 
security,  and  accomplishing  missions.  The  concept  of  a  GS  frame  does  not  mean  police  or 
military  can,  or  should,  abandon  their  other  frames.  There  will  be  times  when  police  and 
military  need  to  escalate  to  the  use  of  force  to  maintain  control  of  others  (Pinizzotto,  Davis, 
&  Miller,  2006). 

Each  frame  (e.g.,  security,  mission  accomplishment,  GS)  affects  cues,  expectancies, 
goals,  and  actions.  Some  police  and  warfighters  also  have  a  frame  about  how  to  dominate 
others.  Police  refer  to  such  officers  as  ‘badge-heavy’;  the  military  tries  to  keep  them  away 
from  civilians.  This  frame  to  dominate  and  intimidate  others  is  inconsistent  with  the  GS 
frame;  we  list  it  in  Figure  1  because  it  is  a  frame  that  some  police  and  military  have,  and  a 
frame  that  some  GSs  have  had  to  shed. 

We  also  appreciate  that  some  situations  discourage  the  use  of  a  GS  frame.  Police  and 
warfighters  must  put  self-protection  first.  Yet  even  in  these  situations,  GSs  can  sometimes 
find  a  way  to  maintain  their  own  security  without  totally  sacrificing  trust  building.  Police 
officers  may  have  to  arrest  lawbreakers,  but  they  can  make  the  arrest  in  ways  that  are  less 
humiliating. 

One  of  the  factors  that  was  significantly  related  to  the  GS  ratings  was  awareness  of 
long-term  consequences.  The  long-term  perspective  acknowledges  that  security  can  be 
increased  when  one  accepts  a  slightly  greater  immediate  risk  to  gain  trust.  Therefore,  the 
issue  may  not  be  trading  off  security  versus  trust  building,  but  rather  considering 
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long-term  consequences  as  well  as  immediate  motivations.  The  skill  here  is  to  determine 
how  to  take  prudent  risks  while  using  a  GS  frame. 

One  of  our  incidents  involved  a  Marine  captain  who  was  assigned  the  mission  of 
restoring  stability  in  an  Iraqi  town.  He  arranged  a  meeting  with  the  town  leaders,  which 
went  well,  but  as  the  meeting  drew  to  a  close,  he  heard  shots.  He  rushed  outside  and 
learned  that  one  of  his  own  Marines  had  accidentally  discharged  his  weapon  into  the 
peaceful  crowd  gathered  outside  the  meeting  hall.  Three  children  were  wounded,  one 
seriously.  He  had  his  medics  treat  the  wounded.  The  next  day  he  had  another  meeting 
scheduled,  and  he  had  to  decide  how  to  show  up.  Should  he  bring  a  full  contingent  of 
Marines,  for  protection?  Or  should  he  just  bring  the  same  size  unit  that  he  had  the  previous 
day,  so  as  not  to  be  provocative?  He  chose  a  third  course.  He  brought  only  four  other 
Marines,  and  he  walked  to  the  meeting,  carrying  just  a  pistol  concealed  under  his 
protective  vest  and  wearing  a  cap  instead  of  a  helmet.  He  explained  that  he  deliberately 
wanted  to  be  vulnerable  to  reduce  tensions.  He  also  explained  that  his  actions  in  treating 
the  wounded  and  taking  responsibility  for  the  tragic  accident  helped  him  to  gain  trust. 
The  incident  actually  worked  in  his  favour  because  of  the  way  he  handled  it. 


Perspective  taking 

To  effectively  take  on  a  GS  frame,  the  warfighter  will  need  to  be  able  to  take  the 
perspective  of  others.  The  skilled  GS  can  grasp  what  is  motivating  others  and  how  these 
others  are  making  sense  of  a  situation.  They  can  also  anticipate  how  the  civilians  are  likely 
to  react.  The  skill  of  perspective  taking  seems  essential  to  taking  on  the  GS  frame,  and  our 
data  showed  a  significant  relationship  between  them.  Perspective  taking  allows  genuine 
respect  rather  than  contempt  because  the  GS  will  appreciate  that  there  is  a  reason  for 
unusual  behaviours.  It  permits  genuine  empathy  because  it  acknowledges  the  goals  of 
others.  Field  Manual  6-22  Army  Leadership  recognizes  empathy  as  a  critical  attribute  to 
leadership  and  a  useful  tool  to  win  support  of  a  population  (U.S.  Army,  2006). 

One  police  officer  that  happened  to  be  head  of  a  SWAT  unit  described  an  encounter 
with  a  man  who  had  stabbed  another  man  and  was  barricaded  inside  his  house.  The  officer 
decided  to  try  to  get  the  man  to  come  out  peacefully.  The  man  shouted  that  he  was  not 
coming  out  -  He  knew  the  police  were  just  going  to  throw  him  to  the  ground.  The  police 
officer  realized  he  now  had  an  opening.  He  told  the  man  that  they  would  not  throw  him  to 
the  ground.  If  he  followed  directions  and  came  out  peacefully,  they  would  handcuff  him 
but  would  not  force  him  to  the  ground.  Relieved,  the  man  did  agree  to  come  out  and  was 
arrested  quietly  and  without  incident. 

The  GSs  try  to  get  into  another  person’s  head,  to  feel  what  another  might  feel  and  to 
sense  what  the  person  would  see  as  a  satisfying  resolution.  If  GSs  can  imagine  how  it 
would  feel  if  people  were  to  push  their  way  into  their  own  homes,  they  would  be  more 
likely  to  seek  a  less  offensive  way  to  carry  out  their  missions.  For  example,  one  soldier 
explained  how  he  managed  interactions  respectfully,  ‘When  I  worked  a  check  stop,  I  try 
to  keep  in  mind  how  it  would  feel  to  have  a  stranger  from  a  foreign  country  stopping  me 
and  demanding  my  identification’. 

Culture  imposes  limits  on  perspective  taking.  Military  interviewees  often  arrived  in  a 
country  lacking  knowledge  of  cultural  variation.  They  might  understand  the  universal 
imperative  of  demonstrating  respect  but  not  the  culture-specific  ways  of  doing  this. 
Assuming  Western  signs  of  respect  to  be  universal  can  convey  the  wrong  message.  Direct 
eye  contact  may  indicate  respect  in  one  place,  while  in  another,  one  should  divert  one’s 
gaze  to  convey  respect.  Warfighters  cannot  readily  put  themselves  in  the  other  person’s 
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shoes  (which  is  a  standard  technique  in  perspective  taking  when  dealing  with  one’s  own 
culture).  The  warfighter  has  to  really  stretch  to  imagine  how  foreigners  are  making  sense 
of  a  situation. 

Perspective  taking  often  depends  on  careful  observation  to  gather  information  on  the 
other’s  goals,  fears,  and  values.  It  depends  on  understanding  how  status  differences 
(gender,  age,  kinship,  power  status)  contribute  to  the  situation.  Recently,  some  research 
has  examined  factors  supporting  multicultural  perspective  taking  (Rentsch,  Gunderson, 
Goodwin,  &  Abbe,  2007),  and  ways  to  increase  perspective  taking  skills  (Gehlbach, 
Young,  &  Roan,  2012;  Roan  et  al.,  2009).  There  is  also  some  literature  on  the  process  of 
gaining  empathy  with  others  to  understand  what  they  think  and  feel  (Ickes,  2003),  but  we 
did  not  locate  any  guidance  for  police/military  personnel.  Klein  (2004;  Klein  &  Kuperman, 
2008)  described  a  cultural  training  approach,  the  cultural  lens  model,  which  is  specifically 
aimed  at  helping  people  gain  the  perspective  of  someone  from  another  culture. 

In  trying  to  understand  why  another  person  has  acted  in  an  unexpected  way,  a  GS 
might  consider  a  few  factors:  The  person  may  have  a  different  motivation;  the  person  may 
have  a  different  set  of  priorities;  the  person  may  be  operating  under  a  different  set  of 
constraints;  and  there  may  be  a  mismatch  in  knowledge  -  The  person  may  lack  some 
knowledge  or,  conversely,  the  person  may  have  knowledge  that  the  GS  lacks. 


Gaining  rapport 

The  skill  of  gaining  rapport  with  others  seems  very  useful  for  building  trust  and  for 
voluntary  compliance  and  cooperation.  It  was  significantly  correlated  with  the  GS  ratings. 
Rapport  covers  a  variety  of  subskills.  Collaborators  on  the  DARPA  SSIM  programme 
identified  a  set  of  important  rapport-building  behaviours  (Damari  &  Logan-Terry,  2015), 
and  these  are  listed  below: 

•  Acknowledging  the  goals  of  the  civilians.  Where  appropriate,  helping  to  pursue  these 
goals, 

•  Searching  for  shared  goals.  Where  appropriate,  adding  those  goals  to  yours, 

•  Establishing  and  demonstrating  empathy, 

•  Getting  civilians  to  like  you, 

•  Promoting  common  ground,  and  repairing  common  ground  when  it  has  been  breaking 
down, 

•  Demonstrating  respect  for  civilians,  as  opposed  to  contempt  for  the  way  they  conduct 
themselves, 

•  Having  a  positive  attitude  and  orientation  towards  civilians, 

•  Showing  appreciation  and  offering  praise, 

•  Refusing  to  personalize  insults, 

•  Trying  to  remain  calm, 

•  Using  social  gaze  -  Maintaining  eye  contact  as  a  sign  of  connection  except  in  cultures 
where  prolonged  eye  contact  creates  discomfort  or  is  seen  as  rude, 

•  Being  able  to  listen  well, 

•  Being  familiar  with  rituals  such  as  greetings,  farewells,  and  thank  yous,  and 

•  Being  ready  to  engage  with  strangers. 

During  this  project,  we  heard  about  the  ‘chat-up  man’  identified  in  British  Army  units  - 
The  soldier  that  others  turn  to  for  meeting  and  connecting  with  unfamiliar  civilians.  Such 
soldiers  reduce  fear  and  offer  reassurance  to  nervous  populations. 
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Voluntary  compliance  and  cooperation 

The  skill  here  is  to  bend  the  person  to  your  will  without  using  force  or  threats.  This  is  the 
power  of  influence  and  persuasion.  It  goes  beyond  the  simple  formulae  of  verbal  judo,  as 
used  by  police  officers  (Thompson  &  Jenkins,  1993/2004).  Voluntary  compliance  taps 
into  the  work  of  Cialdini  and  colleagues  (Cialdini,  1993;  Goldstein,  Martin,  &  Cialdini, 
2008)  on  influence  and  persuasion.  It  is  a  key  method  for  implementing  the  GS  frame 
because  military  and  police  have  to  get  people  to  choose  to  do  what  they  do  not  want  to. 

We  identified  a  range  of  different  approaches  to  gaining  voluntary  compliance.  These 
include  the  work  of  Cialdini  and  colleagues  on  influence  (Cialdini,  1993;  Goldstein  et  al. , 
2008),  Amdur  (2011),  Carnegie  (1936),  Glennon  (2010),  Haley  (1963),  Klein  (2009), 
Thaler  and  Sunstein  (2008),  Thompson  and  Jenkins  (1993/2004),  and  Wilson  (201 1).  One 
of  the  striking  features  of  this  literature  is  that  with  very  few  exceptions,  none  of  the 
authors  cites  any  of  the  others.  This  is  a  very  fragmented  literature,  suggesting  an 
opportunity  to  try  to  synthesize  the  different  traditions  (Klein,  2013)  into  a  general 
account  of  voluntary  compliance.  Weinschenk  (2013)  has  provided  a  good  summary  of 
voluntary  compliance  methods. 


De-escalation 

Some  of  the  incidents  showed  impressive  mastery  of  de-escalation  in  difficult  situations. 
For  example,  a  military  participant  described  an  incident  during  a  tense  period,  when 
rumours  started  that  a  shrine  in  a  neighbouring  district  had  been  desecrated  and 
worshippers  mistreated.  An  angry  crowd  moved  towards  the  military  base.  The  officer 
in  charge  decided  to  engage  the  crowd  rather  than  confronting  it.  He  directed  his 
subordinate  Marines  to  stand  by  while  he,  his  translator,  and  a  senior  civilian  aid  worker 
made  the  encounter  with  the  crowd.  He  was  able  to  separate  the  leaders  of  the  crowd, 
whom  he  knew,  and  to  ask  them  to  sit  down  and  talk.  He  explained  that  he  did  not 
believe  these  rumours  to  be  true  but  that  he  would  have  them  investigated  and  report 
back  to  them.  As  their  discussions  lasted  a  long  time,  the  crowd  dispersed.  He  quickly 
followed  up  by  investigating  the  accusations  and  confirming  their  falsehood  with 
photographs.  In  the  following  week,  he  brought  the  pictures  to  all  of  the  leaders 
involved  in  the  initial  confrontation  and  discussion.  The  interviewee  indicated  that  there 
had  been  no  further  trouble  in  that  region  during  the  remainder  of  his  deployment 
there. 

The  GS  frame  encourages  the  use  of  de-escalation  tactics  to  lower  emotional  affect 
before  tensions  arise  out  of  hand.  The  interviews  suggested  four  strategies  for  de- 
escalating  conflicts.  First,  military  personnel  need  to  remain  cognizant  of  the  trade-off 
between  trust  building  and  prudent  risk.  Police  have  learned  how  dangerous  it  is  when 
they  intervene  in  domestic  violence  incidents  to  try  to  de-escalate  the  fight.  Second,  the 
earlier  a  de-escalation  effort  is  initiated,  the  greater  the  chance  of  success.  Conflicts  may 
have  tipping  points,  after  which  de-escalation  is  more  difficult.  Third,  perspective  taking 
provides  tools  for  attempting  to  understand  the  dynamics  underlying  the  escalation.  If,  for 
example,  the  growing  anger  stems  from  differences  in  motivation,  it  may  be  possible  to 
reframe  the  situation  to  highlight  advantages  for  both  sides.  It  will  be  important  to  look  for 
different  constraints  and/or  different  knowledge  that  have  created  such  resulting  anger. 
Finally,  rapport-building  activities  can  prevent  and  break  the  cycle  of  hostility.  For 
example,  it  will  be  important  to  remain  calm  and  listen  carefully.  It  will  also  be  important 
to  avoid  taking  insults  personally. 
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Conclusions 

The  purpose  of  our  study  was  to  gain  a  better  understanding  of  what  makes  police  and 
military  who  are  GSs  so  effective.  We  achieved  that  goal  with  the  concept  of  a  GS  frame  of 
seeking  to  build  trust,  distinct  from  alternative  frames,  using  the  data/frame  model  of 
sensemaking  to  explain  professional  identity. 

We  relied  on  a  cognitive  task  analysis  method,  a  qualitative  approach  better  suited  to 
generating  descriptions  and  hypotheses  than  to  testing  hypotheses  and  providing 
quantitative  evidence.  We  believe  our  findings  have  potential  value  for  identifying  and 
training  GSs. 

We  formulated  a  sensemaking  account  that  views  GSs  as  seeking  to  gain  the  trust  of 
civilians,  as  shown  in  Figure  1.  Note  that  we  are  not  suggesting  that  GSs  increase  their 
trust  in  civilians  -  There  is  no  reason  to  make  GSs  unnecessarily  vulnerable. 

We  also  considered  whether  any  of  the  other  skills  we  examined  might  serve  as  the  top 
frame,  but  our  opinion  is  that  none  of  the  others  has  the  same  capacity  for  organizing  and 
directing  behaviour.  It  is  not  enough  for  GSs  to  be  trustworthy.  They  have  to  demonstrate 
their  trustworthiness  to  civilians.  They  have  to  perform  actions  and  in  other  ways  convey 
to  civilians  that  it  makes  sense  to  trust  these  warfighters  or  police. 

Being  able  to  take  someone  else’s  perspective  is  a  pre-condition  for  demonstrating 
trustworthiness,  but  it  is  not  directly  communicable.  That  is  why  perspective  taking  is  a 
supporting  skill  for  gaining  trust.  If  rapport-building  was  the  top  frame,  police  and 
military  might  ask  why  -  What  is  the  purpose  of  the  rapport?  If  voluntary  compliance  was 
the  top  frame,  then  it  would  fail  to  address  situations  that  do  not  involve  compliance  with 
requests,  such  as  civilians  proactively  offering  information  and  suggesting  cooperation.  If 
de-escalation  was  the  top  frame,  it  would  fail  to  address  situations  that  do  not  involve 
interpersonal  conflicts.  Our  conclusion  was  that  the  frame  of  seeking  to  build  civilian  trust 
was  the  most  useful  for  organizing  behaviours  and  guiding  the  actions  of  police  and 
military.  Certainly,  the  issue  of  trust  has  come  up  many  times  before  but  never,  to  our 
knowledge,  in  the  form  of  a  sensemaking  frame  that  is  different  from  other  frames  (for 
security,  mission  accomplishment,  compliance  with  regulations,  or  exerting  dominance 
over  others). 


Caveats 

In  all  but  one  case,  we  interviewed  police  and  military  personnel  identified  as  GSs,  so  we 
do  not  have  a  basis  for  speculating  about  the  wider  population  of  police  and  military  other 
than  through  the  comments  of  the  interviewees  about  their  colleagues  and  about  other 
units. 


Next  steps 

One  follow-on  study  would  be  to  examine  a  broader  population  of  military  personnel,  not 
hand-picked  for  having  GS  qualities.  We  are  currently  in  the  process  of  conducting  that 
study.  To  train  GSs,  it  is  not  enough  to  determine  what  skills  and  frame  they  need  -  It  is  also 
necessary  to  see  whether  those  skills  are  missing  in  the  general  training  population. 

A  second  follow-on  study  would  attempt  to  train  military  personnel  to  acquire  a  GS 
frame  or  to  strengthen  such  a  frame.  We  are  also  in  the  process  of  conducting  that  study, 
using  the  ShadowBox  training  method  (Klein,  Hintze,  &  Saab,  2013).  We  are  also 
developing  a  training  module  for  one  of  the  enabling  skills  shown  in  Figure  1,  gaining 
voluntary  compliance  and  cooperation. 
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A  third  potential  direction  for  follow-on  research  would  be  to  explore  the  notion  of 
sensemaking  frames  for  professional  identity  in  many  specialties;  areas  other  than  police 
or  military  work. 

A  fourth  potential  direction  is  to  test  some  of  the  cognitive  frames  listed  in  Figure  1 .  If  it 
was  possible  to  independently  identify  police  or  military  as  relying  on  a  GS  frame  versus 
one  of  the  other  frames  (e.g.,  to  dominate  others,  to  maintain  safety,  to  adhere  to 
regulations),  then  it  should  be  possible  to  predict  their  behaviour  when  shown 
challenging  scenarios.  The  professional  frame  they  use  should  predict  what  cues  they 
notice,  what  goals  they  prioritize,  what  they  expect  to  happen  next,  and  what  response 
options  they  favour.  Just  as  people  use  their  frames  to  understand  the  world,  we  should  be 
able  to  use  our  knowledge  of  their  frames  to  understand  them. 

One  final  contribution  of  this  research,  with  potential  for  follow-on  work,  is  the 
expansion  of  NDM  theory  and  methodology.  Instead  of  just  cataloguing  a  list  of  GS  skills, 
we  chose  to  capture  a  GS  sensemaking  frame.  We  also  expanded  our  concept  of  a 
sensemaking  frame,  drawing  on  the  RPD  model,  to  identify  four  primary  slots, 
expectancies,  cues,  goals,  and  actions.  For  future  work,  we  might  use  those  slots,  and 
the  concept  of  alternative  frames,  in  conducting  interviews  and  gathering  data.  Our  study 
expanded  how  we  view  sensemaking  as  a  framing  process  and  illustrated  the  potential  for 
viewing  a  person’s  professional  identity  in  terms  of  the  way  he/she  makes  sense  of 
situations,  in  contrast  to  alternative  frames. 
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